Search String An alys is P r o j e ct Jeff Chen, Chief Data Scientist Star Ying, Data Scientist
Hopes and dreams go here
Go!
search query went here
Go!
Results for: “search query went here” 1. Ideally your best result 2. But just in case, here’s another 3. And another 4. And another 5. And another 6. We can do this all day 7. More to come 8. Here’s another one. 9. This one’s pretty good. 10. Below here, you probably won’t be paying attention 11. …I was wrong…
For search and retrieval to work… Text acquisition
Query parser
Text transformation
Synonym engine
Index creation
Ranking engine
+n
For search and retrieval to work… Text acquisition
Query parser
Text transformation
Space reserved for synonym engine
Index creation
Ranking engine
+n
Problem! Record is as good as the metadata and tags.
Problem!
W.I.T.A
Problem! Terminology evolves like Pokémon. (therefore it’s hard to catch match them all)
Problem!
[Pilot] Solution new ranking engine to recommend terms light tree
christmas
well
flow
holiday
wellhead
Solution logs Search logs 80 million search log records
Solution In Art Area i pr(tree | christmas) pr(christmas | tree)
Parallelizable python program • ETL of logs • Estimate ranked linear conditional probability estimates