Posted on 02/23/2005 12:16:01 PM PST by Technocrat
Now, if you could find a way to eliminate those pesky duplicate REPLIES, ... ;O)
Ah. Anyway, it didn't return any primary or secondary matches at all, and the tertiary matches were pretty weak, which I suppose is to be expected - I assume this is a problem that will resolve itself as the indexes expand, though.
That's right! The article you posted was one that I inserted to the search engine earlier, and provided earlier in this thread as a good example of how you can get related results because the domain set of that article was so rich. Now I indexed the entire source of the article, which was quite extensive compared to the piece you posted, so you will probably see the original copy in the secondary match list.
The fact that you got a 0 match cost on the primary tier means that the engine thinks it has a precise duplicate of something already in the engine (which it does, since you tried to insert it twice).
Unfortunately, I haven't indexed all of FR on this server, because I would need permission from the mighty ones to do that. So far I have a deafening silence, and I'm not quite sure what to do about that.
Ouch. Really. Ouch. :)
Hmmm. I just did exactly the same thing, and the original article (which has a maximum magnitude of over 150) doesn't even appear in the list. I may need to change this so the first paragraphs are indexed the strongest, and succeeding paragraphs are done with less and less emphasis. Another project for this afternoon!
Yes - it would be really cool if we could get permission to get the last 100,000 articles in the DB. I'll work on that once I feel like the quality is up to snuff.
Perhaps you should email Jim and John privately - make it clear that you're willing to sign over your code to them gratis (assuming you're willing to do that, of course), and that the code is in fact your own original work that you are free to give to them. And then wait, I guess.
Anyway, for all we know, John was three days away from deploying his own search engine, so this isn't really useful to them. Or perhaps there are legal issues that they need to check out before accepting it from you. Who knows? In any case, it seems likely to me that even if da boss couldn't use this for some reason, he'd let you know rather than just leave you hanging....
I've always envisioned the 'perfect duplicate-avoiding FR search algorithm' would do exactly that. In fact, I'd have it look for phrase matches (e.g. words 1 thru 6, 2 thru 7, 3 thru 8, etc.) and only looking at first 100 or so words. Seems like that'd expose any potential duplicates for sure. But, my thoughts were always centered on duplicates avoidance, never on finding loosely-related articles.
Domain fingerprint: -------------------------------- Central domains -------------------------------- Middle East 10 medical 8 organization - people 5 -------------------------------- Peripheral Domains: -------------------------------- military 3 finance/economy 2 government 2 law/truth 2 process 2 travel 2 Africa 2 Iraq 2 education 1 factory 1 flaws 1 information 1 books/mags/print media 1 physics 1 politics/elections 1 relationship 1 science 1 sports 1 Western US 1 |
Primary Matches: (strong match on most subjects, match cost under 10 is duplicate) (Groundhog Day? Here's another article on Punxsutawney Phil.) |
Secondary Matches: (significant shared subject matter) (Groundhog Day? Here's an article on rodent-centered holidays in India.) |
Tertiary Matches: (somewhat related on one or two points) (Groundhog Day? Here's some articles on weather, and some groundhog recipes.) |
Domain fingerprint: -------------------------------- Central domains -------------------------------- Middle East 10 medical 8 organization - people 5 International rel 5 -------------------------------- Peripheral Domains: -------------------------------- military 4 finance/economy 2 geography 2 government 2 law/truth 2 mathematics 2 process 2 travel 2 Africa 2 Iraq 2 education 1 factory 1 flaws 1 information 1 books/mags/print media 1 physics 1 politics/elections 1 relationship 1 religion 1 science 1 sports 1 weapons 1 Western US 1 |
Primary Matches: (strong match on most subjects, match cost under 10 is duplicate) (Groundhog Day? Here's another article on Punxsutawney Phil.) |
Secondary Matches: (significant shared subject matter) (Groundhog Day? Here's an article on rodent-centered holidays in India.) Idi Amin is overweight (89) ms madina amin yesterday said that her husband is ailing because of advanced age and overweight. she said that mr idi amin dada, 80, who was yesterday still in coma, weighs about 220 kilogrammes.at his age and then the fact that he is overweight th |
Tertiary Matches: (somewhat related on one or two points) (Groundhog Day? Here's some articles on weather, and some groundhog recipes.) dozens of cia operatives killed (383) the shocking story surfaced on feb. 2, when former pentagon adviser richard n. perle told the house intelligence committee about what he called the terrible setback that we suffered in iran a few years ago when, in a display of unbelievable, careles cnn news chief quits over iraq remarks (islam onlines take, al-barf alert) (391) after 23 years at cnn, i have decided to resign in an effort to prevent cnn from being unfairly tarnished by the controversy over conflicting accounts of my recent remarks regarding the alarming number of journalists killed in iraq, jordan said in a syria, lebanon and flying pigs (424) skip to comments. syria, lebanon and flying pigs jinsa ^ | february 22, 2005 3:09:36 pm pst by ooh-ah the times of london reports that syria has agreed to remove its 14,000 troops from lebanonsoon as it had agreed to do in the 1989 taif accord, pr iranian alert - october 19, 2004 [est]- iran live thread - americans for regime change in iran (429) he us media still largely ignores news regarding the islamic republic of iran. as tony snow of the fox news network has put it, this is probably the most under-reported news story of the year. as a result, most americans are unaware that the islamic navy medicine will protect service members, patients despite flu vaccine shortages (451) department of defense officials said their supplies for all the services are less than expected about 1.5 million fewer doses than projected. however, those who most need the vaccine will get it, according to capt. edward m. kilbane, an infectious d nra bashs u.n. on small-arms reduction (524) wayne lapierre, executive vice president of the national rifle association of america, has used the conservative political actions conference as a forum to harshly criticize u.n. efforts to reduce the amount of small arms in the hand of civilians. a open letter to the citizens of the united states of america (mega-triple-barf alert!!) (525) as a journalist who has the good fortune to write for an international journal... (snip) ...i write this letter as a citizen of this international community and as a journalist for a newspaper whose name is pravda (truth), i have the obligation to te (614) seismic changes: studies indicate major land shifts bankok and phuket moved) the nation (bangkok) ^ | february 23, 2005 12:04:12 pm pst by nickcarraway sumatra quake pushed phuket 32cm southwest and moved bangkok: expert phuket shifted 32 centimetr |
It sounds like it just means there are 150 different variables that they define for cataloging it all.
That makes more sense.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.