id
the search is over
Christopher M. Judd
Christopher M. Judd CTO and Partner at leader Developer User Group (CIDUG) BROUGHT TO YOU BY:
C O NT E NT S
Get More Refcardz! Visit dzone.com/refcardz
221
Getting Started With Docker
» About Docker » Docker Architecture » Getting Started » Typical Local Workflow » Other Helpful Commands
By Christopher M. Judd
» Dockerfile, and more...
A B O U T D O CK E R Almost overnight, Docker has become the de facto standard that developers and system administrators use for packaging, deploying, and running distributed applications. It provides tools for simplifying DevOps by enabling developers to create templates called images that can be used to create lightweight virtual machines called containers, which include their applications and all of their applications’ dependencies. These lightweight virtual machines can be promoted through testing and production environments where sysadmins deploy and run them. Docker Images
Docker makes it easier for organizations to automate infrastructure, isolate applications, maintain consistency, and improve resource utilizations.
A recipe or template for creating Docker containers. It includes the steps for installing and running the necessary software.
Docker Like a tiny virtual machine that is created from Container the instructions found within the Docker image originated
Similar to the popular version control software Git, Docker has a social aspect, in that developers and sysadmins are able to share their images via
Docker Client
Command-line utility or other tool that takes advantage of the Docker API (https://docs.docker. com/reference/api/docker_remote_api) to communicate with a Docker daemon
Docker Host
A physical or virtual machine that is running a Docker daemon and contains cached images as well as runnable containers created from images
Docker Hub.
G E T T I N G S TA R T E D W I T H D O C K E R
Columbus
Docker is an open-source solution that runs natively on Linux but also works on Windows and Mac using a lightweight Linux distribution and VirtualBox. Many
Site24x7 DOCKER MONITORING
tools have also grown up around Docker to make it easier to manage and orchestrate complex distributed applications.
Get Detailed Insight into Docker Containers
DO CK ER A RCHITEC TU R E Docker utilizes a client-server architecture and a remote API to manage and create Docker containers built upon
CPU
Docker images. The relationship between containers and images are analogous to the relationship between objects and classes in object-oriented programming.
© DZONE, INC.
Cache
Memory
Linux containers. Docker containers are created from
|
DZONE.COM
I/O
searching is easy right?
select * from products where name = ‘iPhone 5’
select * from products where description like ‘%iphone%’
select * from products where match(name, description) against(‘+iphone -case’ in boolean mode);
http://lucene.apache.org/
Users%
Search'User'Interface' Build& Query&
Render& Result&
Run$Query$
Index&
Index&Document& Analyze(Document(
Build&Document&
Acquire(Document( Raw$ Content$
public class InMemoryExample { public static void main(String[] args) throws CorruptIndexException, LockObtainFailedException, IOException, ParseException { // in-memory representation of the index RAMDirectory idx = new RAMDirectory(); // Make an writer to create the index IndexWriterConfig config = new IndexWriterConfig(LUCENE_36, new StandardAnalyzer(LUCENE_36)); IndexWriter writer = new IndexWriter(idx, config); // Add some Document objects containing quotes writer.addDocument(createDocument( "Theodore Roosevelt", "It behooves every man to remember that the work of the " + "critic, is of altogether secondary importance, and that, " + "in the end, progress is accomplished by the man who does " + "things.")); writer.addDocument(createDocument( "Friedrich Hayek", "The case for individual freedom rests largely on the " + "recognition of the inevitable and universal ignorance " + "of all of us concerning a great many of the factors on " + "which the achievements of our ends and welfare depend.")); writer.addDocument(createDocument( "Ayn Rand", "There is nothing to take a man's freedom away from " + "him, save other men. To be free, a man must be free " + "of his brothers.")); writer.addDocument(createDocument( "Mohandas Gandhi", "" + "Freedom is not worth having if it does not connote " + "freedom to err.")); writer.close(); // Build an IndexSearcher using the in-memory index IndexReader reader = IndexReader.open(idx); IndexSearcher searcher = new IndexSearcher(reader); // Run some queries search(searcher, "freedom"); search(searcher, "free"); search(searcher, "progress or achievements"); searcher.close(); }
search(searcher, "progress or achievements"); searcher.close(); } /** * Make a Document object with an un-indexed title field and an indexed content field. */ private static Document createDocument(String title, String content) { Document doc = new Document(); // Add the title as an unindexed field... doc.add(new Field("title", title, Field.Store.YES, Field.Index.NOT_ANALYZED)); doc.add(new Field("content", content, Field.Store.YES, Field.Index.ANALYZED)); return doc; } /** * Searches for the given string in the "content" field */ private static void search(IndexSearcher searcher, String queryString) throws ParseException, IOException { // Build a Query object Query query = new QueryParser(LUCENE_36, "content", new StandardAnalyzer(LUCENE_36)).parse(queryString); TopDocsCollector collector = TopScoreDocCollector.create(10, true); searcher.search(query, collector); if (collector.getTotalHits() == 0) { System.out.println("No matches were found for \"" + queryString + "\""); } else { System.out.println("Hits for \"" + queryString + "\" were found in quotes by:"); ScoreDoc[] hits = collector.topDocs().scoreDocs; for (ScoreDoc hit : hits) { Document doc = searcher.doc(hit.doc); System.out.println(" - " + doc.get("title")); } } System.out.println(); } }
search(searcher, "progress or achievements"); searcher.close(); } /** * Make a Document object with an un-indexed title field and an indexed content field. */ private static Document createDocument(String title, String content) { Document doc = new Document(); // Add the title as an unindexed field... doc.add(new Field("title", title, Field.Store.YES, Field.Index.NOT_ANALYZED)); doc.add(new Field("content", content, Field.Store.YES, Field.Index.ANALYZED)); return doc; } /** * Searches for the given string in the "content" field */ private static void search(IndexSearcher searcher, String queryString) throws ParseException, IOException { // Build a Query object Query query = new QueryParser(LUCENE_36, "content", new StandardAnalyzer(LUCENE_36)).parse(queryString); TopDocsCollector collector = TopScoreDocCollector.create(10, true); searcher.search(query, collector); if (collector.getTotalHits() == 0) {Hits for "freedom" were found in quotes System.out.println("No matches were found for \"" + queryString + "\""); - Mohandas Gandhi } else { System.out.println("Hits for \"" + queryString Ayn Rand + "\" were found in quotes by:");
by:
- Friedrich Hayek
ScoreDoc[] hits = collector.topDocs().scoreDocs; for (ScoreDoc hit : hits) { Document doc = searcher.doc(hit.doc); Hits for "free" System.out.println(" - " + doc.get("title")); - Ayn Rand } } System.out.println(); } }
were found in quotes by:
Hits for "progress or achievements" were found in quotes by: - Theodore Roosevelt - Friedrich Hayek
Index Document Field Value title
Field
Term
Value
Term
Document Field Value
Term
content
title
Field content
…
Value
Term
Theodore Roosevelt
It
Term
behooves
Term
every
…
case
Term
for
…
Fredrich Hayek
The
Term
Documents 1: Theodore Roosevelt It behooves every man to remember that the work of the critic, is of altogether secondary importance, and that, in the end, progress…
Term Index
… progress
…
1
achievements
1
free
3
freedom
… …
2
2: Friedrich Hayek
2
3
The case for individual freedom rests largely on the recognition of the inevitable and universal ignorance of all of us concerning …
3: Ayn Rand There is nothing to take a man's freedom away from him, save other men. To be free, a man must be free of his brothers.
popular powerful
Java only single threaded writes lots of boiler plate code difficult to debug and test indexes no user interface
http://lucene.apache.org/solr/
scalable caching highlighting boosting facets clustering database integration rich document (Word, PDF) indexing geospatial search REST-like HTTP/XML and JSON API
Application
datasources
bin/solr start -e cloud Welcome to the SolrCloud example! To begin, how many Solr nodes would you like to run in your local cluster? (specify 1-4 nodes) [2] Please enter the port for node1 [8983] Please enter the port for node2 [7574] Starting up SolrCloud node1 on port 8983 using command: solr start -cloud -s example/cloud/node1/solr -p 8983
-m 512m
Waiting to see Solr listening on port 8983 [/] Started Solr server on port 8983 (pid=31995). Happy searching! Starting node2 on port 7574 using command: solr start -cloud -s example/cloud/node2/solr -p 7574 -z localhost:9983 -m 512m Waiting to see Solr listening on port 7574 [/] Started Solr server on port 7574 (pid=32086). Happy searching! Now let's create a new collection for indexing documents in your 2-node cluster. Please provide a name for your new collection: [gettingstarted] workshop workshop How many shards would you like to split workshop into? [2] 2 How many replicas per shard would you like to create? [2] 2 Please choose a configuration for the workshop collection, available options are: basic_configs, data_driven_schema_configs, or sample_techproducts_configs [data_driven_schema_configs] sample_techproducts_configs Connecting to ZooKeeper at localhost:9983
Creating new collection 'workshop' using command: http://172.25.5.134:7574/solr/admin/collections?action=CREATE&name=workshop&numShards=2&replicationFactor=2&maxShardsPerNode=2&collection.con {“responseHeader”:{“status”:0,"QTime":4017}, “success”:{“”:{"responseHeader":{"status":0,"QTime":3807},"core":"workshop_shard2_replica1"}}} Enabling auto soft-commits with maxTime 3 secs using the Config API POSTing request to Config API: http://localhost:8983/solr/workshop/config {"set-property":{"updateHandler.autoSoftCommit.maxTime":"3000"}} Successfully set-property updateHandler.autoSoftCommit.maxTime to 3000 SolrCloud example running, please visit http://localhost:8983/solr
http://localhost:8983/solr/
Stop bin/solr stop -all Restart bin/solr start -e cloud -noprompt
Setup Lab
1. Install Solr >= 5.2.1 2. Start Solr 3. Browse to http://localhost:8983/solr/
Basic Querying
Users%
Search'User'Interface' Build& Query&
Render& Result&
Run$Query$
Index&
request handler
output format
http://localhost:8983/solr/gettingstarted_shard1_replica1/select?q=*%3A*&wt=json
index
query for everything
{"responseHeader":{"status":0,"QTime":14,"params": {"q":"*:*","wt":"json"}},"response":{"numFound":0,"start": 0,"maxScore":0.0,"docs":[]}}
/select?q=*:*&wt=json&indent=on pretty print
{ "responseHeader":{ "status":0, "QTime":4, "params":{ "q":"*:*", "indent":"true", "wt":"json"}}, "response":{"numFound":0,"start":0,"maxScore":0.0,"docs":[] }}
/select?q=*:*&wt=xml&indent=on xml
0 6 <str name="q">*:* <str name="wt">xml
Index Data
Index&
Index&Document& Analyze(Document(
Build&Document&
Acquire(Document( Raw$ Content$
JSON XML CSV Database
https://developers.google.com/books/docs/v1/getting_started
<doc> false name="description">Written with a rare combination of analysis and
…
[ { "id": "ka2VUBqHiWkC", "title": "Effective Java", "author": "Joshua Bloch", "author_txt": [ "Joshua Bloch" ], "page_i": 368, "saleable_b": true, "description": "Are you looking for a deeper understanding of the Java …”, "price": 25.49 }, { "id": "-SYM4PW-YAgC", "title": "The Religion of Java", "author": "Clifford Geertz", "author_txt": [ "Clifford Geertz" ], "page_i": 392, "saleable_b": false, "description": "Written with a rare combination of analysis and speculation …” } ]
java -Dc=workshop -Dtype=application/json -jar post.jar books/googlebooks*.json
SimplePostTool version 5.0.0 Posting files to [base] url http://localhost:8983/solr/workshop/update using contenttype application/json... POSTing file googlebooks.json to [base] POSTing file googlebooks1.json to [base] POSTing file googlebooks2.json to [base] 3 files indexed. COMMITting Solr index changes to http://localhost:8983/solr/workshop/update... Time spent: 0:00:00.175
Index Lab
1. 2. 3. 4. 5. 6.
Load Google book data Perform a search to determine how many books Return a result in XML Return a human readable result in XML Return a result in JSON Return a human readable result in JSON
More Querying
/select?q=*:*&wt=json&indent=true { "responseHeader": { "status": 0, "QTime": 0, "params": { "indent": "true", "q": "*:*", "_": "1403710200465", "wt": "json" } }, "response": { "numFound": 1, "start": 0, "docs": [ { "id": "ka2VUBqHiWkC", "title": [ "Effective Java" ], "author": "Joshua Bloch", "author_s": "Joshua Bloch", "author_txt": [ "Joshua Bloch" ], "saleable_b": true, "description": "Are you looking for a deeper understanding of the Java ...", "_version_": 1471896715487346700 } ] } }
Limit Fields /select?q=*:*&fl=name id&wt=json&indent=true {
"responseHeader":{ "status":0, "QTime":0, "params":{ "fl":"name id", "indent":"true", "q":"*:*", "wt":"json"}}, "response":{"numFound":10,"start":0,"docs":[ { "id":"ka2VUBqHiWkC", "name":"Effective Java"}, { "id":"-SYM4PW-YAgC", "name":"The Religion of Java"}, { "id":"mB_92VqJbsMC", "name":"Java Threads"}, { "id":"Ql6QgWf6i7cC", "name":"Thinking in Java"}, { "id":"zuGy-V3Nk4AC", "name":"Java SE 7 Programming Essentials"}, { "id":"mvzgNSmHEUAC", "name":"Java in a Nutshell"}, { "id":"gJEC2q7DzpQC", "name":"The History of Java"}, { "id":"vvg7fN_HScAC", "name":"Advanced Java Networking"}, { "id":"pnwTLvCJKh0C", "name":"Java Programming"}, { "id":"Y0lDBsh7J9kC", "name":"Learning Java"}] }}
Limit Results /select?q=*:*&rows=5&fl=name+id&wt=json&indent=true
{
"responseHeader":{ "status":0, "QTime":0, "params":{ "fl":"name id", "indent":"true", "q":"*:*", "wt":"json", "rows":"5"}}, "response":{"numFound":10,"start":0,"docs":[ { "id":"ka2VUBqHiWkC", "name":"Effective Java"}, { "id":"-SYM4PW-YAgC", "name":"The Religion of Java"}, { "id":"mB_92VqJbsMC", "name":"Java Threads"}, { "id":"Ql6QgWf6i7cC", "name":"Thinking in Java"}, { "id":"zuGy-V3Nk4AC", "name":"Java SE 7 Programming Essentials"}] }}
Pagination /select?q=*:*&start=5&rows=5&fl=name+id&wt=json&indent=true
{
"responseHeader":{ "status":0, "QTime":0, "params":{ "fl":"name id", "indent":"true", "start":"5", "q":"*:*", "wt":"json", "rows":"5"}}, "response":{"numFound":10,"start":5,"docs":[ { "id":"mvzgNSmHEUAC", "name":"Java in a Nutshell"}, { "id":"gJEC2q7DzpQC", "name":"The History of Java"}, { "id":"vvg7fN_HScAC", "name":"Advanced Java Networking"}, { "id":"pnwTLvCJKh0C", "name":"Java Programming"}, { "id":"Y0lDBsh7J9kC", "name":"Learning Java"}] }}
Query Field /select?q=name:Java&fl=name+id&wt=json&indent=true {
"responseHeader":{ "status":0, "QTime":8, "params":{ "q":"name_txt:java", "indent":"true", "fl":"name id", "wt":"json"}}, "response":{"numFound":10,"start":0,"maxScore":0.51104903,"docs":[ { "id":"mB_92VqJbsMC", "name":["Java Threads"]}, { "id":"ka2VUBqHiWkC", "name":["Effective Java"]}, { "id":"pnwTLvCJKh0C", "name":["Java Programming"]}, { "id":"Y0lDBsh7J9kC", "name":["Learning Java"]}, { "id":"-SYM4PW-YAgC", "name":["The Religion of Java"]}, { "id":"mvzgNSmHEUAC", "name":["Java in a Nutshell"]}, { "id":"gJEC2q7DzpQC", "name":["The History of Java"]}, { "id":"vvg7fN_HScAC", "name":["Advanced Java Networking"]}, { "id":"Ql6QgWf6i7cC", "name":["Thinking in Java"]}, { "id":"zuGy-V3Nk4AC", "name":["Java SE 7 Programming Essentials"]}] }}
Query Field /select?q=author:Bloch&wt=json&indent=true {
"responseHeader":{ "status":0, "QTime":9, "params":{ "q":"author:Bloch", "indent":"true", "wt":"json"}}, "response":{"numFound":1,"start":0,"maxScore":1.1976817,"docs":[ { "id":"ka2VUBqHiWkC", "name":"Effective Java", "author":"Joshua Bloch", "author_s":"Joshua Bloch", "author_txt":["Joshua Bloch"], "pages_i":368, "saleable_b":true, "description":"Are you looking for a deeper understanding of the Java™ programming language so that you can write code tha is clearer, more correct, more robust, and more reusable? Look no further! Effective Java™, Second Edition, brings together seventy-eight indispensable programmer’s rules of thumb: working, best-practice solutions for the programming challenges you encounter every day. This highly anticipated new edition of the classic, Jolt Award-winning work has been thoroughly updated to cover Java SE 5 and Java SE 6 features introduced since the first edition. Bloch explores new design patterns and language idioms, showing you how to make the most of features ranging from generics to enums, annotations to autoboxing. Each chapter in the book consists of several “items” presented in the form of a short, standalone essay that provides specific advice, insight into Java platform subtleties, and outstanding code examples. The comprehensive descriptions and explanations for each item illuminate what to do, what not to do, and why. Highlights include: New coverage of generics, enums, annotations, autoboxing, th for-each loop, varargs, concurrency utilities, and much more Updated techniques and best practices on classic topics, including objects, classes, libraries, methods, and serialization How to avoid the traps and pitfalls of commonly misunderstood subtleties of the language Focus on the language and its most fundamental libraries: java.lang, java.util, and, to a lesser extent, java.util.concurrent and java.io Simply put, Effective Java™, Second Edition, presents the most practical, authoritative guidelines available for writing efficient, well-designed programs.", "cat":["Computers"], "price":25.49, "price_c":"25.49,USD", "_version_":1507586189982433280}] }}
Query Field /select?q=author_s:Bloch&wt=json&indent=true
{ "responseHeader": { "status": 0, "QTime": 17, "params": { "q": "author_s:Bloch", "indent": "true", "wt": "json", "_": "1437746491580" } }, "response": { "numFound": 0, "start": 0, "maxScore": 0, "docs": [] } }
Query Multiple Fields /select?q=name:Java+AND+saleable_b:true &fl=name+id+saleable_b&wt=json&indent=true
{ "responseHeader":{ "status":0, "QTime":9, "params":{ "q":"name:java AND saleable_b:true", "indent":"true", "fl":"name id saleable_b", "wt":"json"}}, "response":{"numFound":6,"start":0,"maxScore":1.3008728,"docs":[ { "id":"mB_92VqJbsMC", "name":["Java Threads"], "saleable_b":true}, { "id":"ka2VUBqHiWkC", "name":["Effective Java"], "saleable_b":true}, { "id":"Y0lDBsh7J9kC", "name":["Learning Java"], "saleable_b":true}, { "id":"mvzgNSmHEUAC", "name":["Java in a Nutshell"], "saleable_b":true}, { "id":"gJEC2q7DzpQC", "name":["The History of Java"], "saleable_b":true}, { "id":"zuGy-V3Nk4AC", "name":["Java SE 7 Programming Essentials"], "saleable_b":true}] }}
Query Ranges /select?q=pages_i:[*+TO+400] &fl=name+id+pages_i&wt=json&indent=true
{
"responseHeader":{ "status":0, "QTime":0, "params":{ "fl":"name id pages_i", "indent":"true", "q":"pages_i:[* TO 400]", "wt":"json"}}, "response":{"numFound":5,"start":0,"docs":[ { "id":"ka2VUBqHiWkC", "name":"Effective Java", "pages_i":368}, { "id":"-SYM4PW-YAgC", "name":"The Religion of Java", "pages_i":392}, { "id":"mB_92VqJbsMC", "name":"Java Threads", "pages_i":340}, { "id":"zuGy-V3Nk4AC", "name":"Java SE 7 Programming Essentials", "pages_i":336}, { "id":"vvg7fN_HScAC", "name":"Advanced Java Networking", "pages_i":399}] }}
Sorting /select?q=pages_i:[*+TO+400]&sort=pages_i+asc &fl=name+id+pages_i&wt=json&indent=true
{
"responseHeader":{ "status":0, "QTime":0, "params":{ "fl":"name id pages_i", "sort":"pages_i asc", "indent":"true", "q":"pages_i:[* TO 400]", "wt":"json"}}, "response":{"numFound":5,"start":0,"docs":[ { "id":"zuGy-V3Nk4AC", "name":"Java SE 7 Programming Essentials", "pages_i":336}, { "id":"mB_92VqJbsMC", "name":"Java Threads", "pages_i":340}, { "id":"ka2VUBqHiWkC", "name":"Effective Java", "pages_i":368}, { "id":"-SYM4PW-YAgC", "name":"The Religion of Java", "pages_i":392}, { "id":"vvg7fN_HScAC", "name":"Advanced Java Networking", "pages_i":399}] }}
Facets
Facets /select?q=*:*&rows=0&wt=json&indent=true &facet=true&facet.field=cat { "responseHeader":{ "status":0, "QTime":35, "params":{ "q":"*:*", "facet.field":"cat", "indent":"true", "rows":"0", "wt":"json", "facet":"true"}}, "response":{"numFound":10,"start":0,"maxScore":1.0,"docs":[] }, "facet_counts":{ "facet_queries":{}, "facet_fields":{ "cat":[ "Computers",8, "Java (Indonesia)",1, "Religion",1]}, "facet_dates":{}, "facet_ranges":{}, "facet_intervals":{}, “facet_heatmaps”:{} } }
Highlighting /select?q=description:java AND name:"Java in a Nutshell"&wt=json&indent=true &hl=true &hl.fl=description &hl.simple.pre=<em> &hl.simple.post=
{
}
"responseHeader":{ "status":0, "QTime":28, "params":{ "indent":"true", "q":"description:java AND title:\"Java in a Nutshell\" ", "hl.simple.pre":"<em>", "hl.simple.post":"<em>", "hl.fl":"description", "wt":"json", "hl":"true"}}, "response":{"numFound":1,"start":0,"docs":[ { "id":"mvzgNSmHEUAC", "title":["Java in a Nutshell"], "author":"David Flanagan", "author_s":"David Flanagan", "author_txt":["David Flanagan"], "page_i":1224, "saleable_b":true, "description":"Aimed for programmers, offers an introduction to Java 5.0 ...", "_version_":1471896715499929600}] }, "highlighting":{ "mvzgNSmHEUAC":{ "description":["Aimed for programmers, offers an introduction to <em>Java 5.0, covering topics such as generics”]} }
Query Lab
1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
Search for all books but only return the id, name and author Return a result with all books Search for any books with java Search for any books without java Search for any books with java in the title (name) Search for any books with java in the title and are available to purchase Search for books between 100 and 200 pages Sort all the books based on the author's name Determine how many books there in each category Highlight the word java in the description of all the books
Configurations
<solr_home>/server/solr/configsets/sample_techproducts_configs/conf
schema.xml
<schema>
stored="true"/> stored="true" multiValued="true"/> stored="true"/> stored="true"/>
id
Users%
Search'User'Interface' Build& Query&
Render& Result&
Run$Query$
Index&
Index&Document& Analyze(Document(
Build&Document&
Acquire(Document( Raw$ Content$
Solr
Query
Response
Request Handler
Response Writer
Lucene index Update Handler
Data Source
solrconfig.xml
<requestHandler name="/select" > <str name="echoParams">explicit 10 <str name="df">text
<requestHandler name="/query" > <str name="echoParams">explicit <str name="wt">json <str name="indent">true <str name="df">text
<requestHandler name="/browse" > <str name="echoParams">explicit
VelocityResponseWriter settings --> name="wt">velocity name="v.template">browse name="v.layout">layout name="title">Solritas
<arr name="last-components"> <str>spellcheck
Configuration Lab
1. 2. 3. 4. 5. 6.
Add a new title field to the schema. Make the title searchable and sortable Restart Solr Reindex Search for Java in the titles Sort titles containing Java in them
http://localhost:8983/solr/workshop/update? stream.body=<delete>*:*
Rich Content
bin/post -c workshop ../solr-data/refcardz/ http://localhost:8983/solr/workshop/select? q=docker&rows=200&wt=json&indent=true { "responseHeader":{ "status":0, "QTime":50, "params":{ "q":"docker", "indent":"true", "wt":"json"}}, "response":{"numFound":1,"start":0,"maxScore":0.05551956,"docs":[ { "links":[ "https://github.com/cloudfoundry", "http://www.refcardz.com", "http://cloudfoundry.org/",], "id":"/Users/cjudd/devl/tmp/solr-5.2.1/../solr-data/refcardz/rc207_010d-cloudfoundry_0.pdf", "resourcename":"/Users/cjudd/devl/tmp/solr-5.2.1/../solr-data/refcardz/rc207_010d-cloudfoundry_0.pdf", "last_modified":"2015-04-15T19:02:41Z", "content_type":["application/pdf"], "content":["\n\n Get Started »\nLearn more: stackato.com/refcard\n \n Deploy apps faster on any cloud with Stackato, the leading\nenterprise PaaS based on Cloud Foundry, Docker and other open\nsource technologies. Free to use in production up to 20GB.\n \n The Easiest Way to \nGet Started with \nCloud Foundry \n \n \n \n \n © DZone, Inc. | DZone.com\n \n cloud Foundry\nopen-Source PaaS for Streamlined Development and Deployment\n \n By Jeremy Voorhis, Revised and Updated by Billy Tat\n \n » Supported Technologies \n \n » Your "], "_version_":1507640053787000832}] }}
bin/post -c workshop http://www.cojug.org -recursive 1 -delay 1
http://localhost:8983/solr/workshop/select? q=*:*&rows=1&wt=json&indent=true {
"responseHeader":{ "status":0, "QTime":31, "params":{ "q":"*:*", "indent":"true", "rows":"1", "wt":"json"}}, "response":{"numFound":23,"start":0,"maxScore":1.0,"docs":[ { "links":["rect", "mainlevel-nav", "http://www.cojug.org/index.php",], "id":"http://www.cojug.org", "url":"http://www.cojug.org", "keywords":"COJUG, Java, Ohio, User Group", "description":"COJUG - Central Ohio Java User's Group", "title":["Central Ohio Java Users Group (COJUG) - Home"], "content_type":["text/html; charset=iso-8859-1"], "content":["\n Central Ohio Java Users Group (COJUG) - Home \n \n Saturday, 25 July 2015\n\nHome\tContact Us\n\tNews ... pageTracker._initData(); \npageTracker._trackPageview();\n \n\n "], "_version_":1507640692579500032}] }}
Geospatial
http://poiplaza.com/
java -Dc=workshop -Dtype=application/json -jar post.jar parks/ amusement_parks.json
[
{
}, {
"id": "Myrtle Waves Water Park", "name": "Myrtle Waves Water Park", "description": "USA-North Myrtle Beach
\nUS 17 Bypass at 10th Avenue
\n", "coordinates_p": "33.81673,-78.68005", "store": "33.81673,-78.68005", "source_s": "USA-Amusement & Theme Parks"
"id": "Coney Island", "name": "Coney Island", "description": "USA-New York-Brooklyn
\n1208 Surf Ave.
\n+1 718-372-5159", "coordinates_p": "40.57546,-73.98017", "store": "40.57546,-73.98017", "source_s": "USA-Amusement & Theme Parks" }, { "id": "Six Flags Over Georgia", "name": "Six Flags Over Georgia", "description": "USA-Austell
\n7561 Six Flags Pkwy
\n+1 770-948-9290", "coordinates_p": "33.77091,-84.55220", "store": "33.77091,-84.55220", "source_s": "USA-Amusement & Theme Parks" } ]
http://localhost:8983/solr/parks/browse
distance filters point distance bounding lat and long
point distance /select?q=*:*&fq={!geofilt}&pt=37.7752,-122.4232&d=100&sfield=coordinates_p&wt=json
point distance filter
{
km
"id": "Myrtle Waves Water Park", "name": "Myrtle Waves Water Park", "description": "USA-North Myrtle Beach< "coordinates_p": "33.81673,-78.68005", "store": "33.81673,-78.68005", "source_s": "USA-Amusement & Theme Park }
bounding latitude and longitude /select?q=*:*&fq={!bbox}&pt=37.7752,-122.4232&d=100&sfield=coordinates_p&wt=json
bounding filter
{
km
"id": "Myrtle Waves Water Park", "name": "Myrtle Waves Water Park", "description": "USA-North Myrtle Beach< "coordinates_p": "33.81673,-78.68005", "store": "33.81673,-78.68005", "source_s": "USA-Amusement & Theme Park }
calculate distance /select?q=*:*&fq={!bbox} &pt=37.7752,-122.4232&d=100&sfield=coordinates_p&wt=json&fl=_dist_:geodist(),name,de scription,coordinates_p&indent=on {
"responseHeader":{ "status":0, "QTime":0, "params":{ "d":"100", "fl":"_dist_:geodist(),name,description,coordinates_p", "indent":"on", "q":"*:*", "sfield":"coordinates_p", "pt":"37.7752,-122.4232", "wt":"json", "fq":"{!bbox}"}}, "response":{"numFound":2,"start":0,"docs":[ { "name":"Six Flags Marine World", "description":"USA-Vallejo
\n2001 Marine World Parkway
\n+1 707-643-6722", "coordinates_p":"38.14176,-122.24957", "_dist_":43.50948310887128}, { "name":"Great America", "description":"USA-Santa Clara
\n2401 Agnew Rd
\n", "coordinates_p":"37.39057,-121.96905", "_dist_":58.57220580622602}] }}
/select?q=*:*&fq={!bbox} &pt=37.7752,-122.4232&d=1000&sfield=coordinates_p&wt=json&fl=_dist_:geodist(),name,d escription,coordinates_p&indent=on&sort=geodist()+asc {
sort by distance
"responseHeader":{ "status":0, "QTime":14, "params":{"d":"1000", "fl":"_dist_:geodist(),name,description,coordinates_p", "sort":"geodist() asc", "indent":"on","q":"*:*","sfield":"coordinates_p","pt":"37.7752,-122.4232", "wt":"json", "fq":"{!bbox}"}}, "response":{"numFound":23,"start":0,"docs":[ { "name":"Six Flags Marine World", "description":"USA-Vallejo
\n2001 Marine World Parkway
\n+1 707-643-6722", "coordinates_p":"38.14176,-122.24957", "_dist_":43.50948310887128}, { "name":"Great America", "description":"USA-Santa Clara
\n2401 Agnew Rd
\n", "coordinates_p":"37.39057,-121.96905", "_dist_":58.57220580622602}, { "name":"Universal Studios Hollywood", "description":"USA-Universal City
\n100 Universal City Plaza
\n+1 800-959-9688", "coordinates_p":"34.13673,-118.35590", "_dist_":545.5031109696113}, { "name":"Santa Monica Pier", "description":"USA-Santa Monica
\n200 Santa Monica Pier
\n+1 310-458-8900", "coordinates_p":"34.01036,-118.49612", "_dist_":547.9619130243483}, { "name":"Raging Waters", "description":"USA-San Dimas
\n111 Raging Waters Drive
\n+1 909-802-2200", "coordinates_p":"34.08565,-117.81186", "_dist_":583.5368017552456}, { "name":"Knott's Berry Farm/Soak City", "description":"USA-Buena Park
\n8039 Beach Blvd.
\n+1 714-220-5200", "coordinates_p":"33.84550,-117.99810", "_dist_":591.5876572669723}, { "name":"Disneyland", "description":"USA-Anaheim
\n700 W Ball Rd
\n+1 714-781-4565", "coordinates_p":"33.81786,-117.91846", "_dist_":598.7499877946933}, {
Geospatial Lab
1. 2. 3.
Clear the current index Load the amusement park data Search for amusement parks near (100 meters) of Orlando Florida (28.405080, -81.579433)
http://www.elasticsearch.org/
ELK collection and processing input | codecs | filters | outputs
indexing and searching
user interface
brought to you by...
Covers Apache Lucene 3.0
IN ACTION
Get More Refcardz! Visit refcardz.com
#120
FOREWORD BY Yonik Seeley
Getting Optimal Search Results
By Chris Hostetter
When LucidWorks is installed at ~/LucidWorks the Solr Home directory is ~/LucidWorks/lucidworks/solr/.
ABOUT SOLR
Single Core and Multicore Setup
Solr makes it easy for programmers to develop sophisticated, high performance search applications with advanced features such as faceting, dynamic clustering, database integration and rich document handling.
By default, Solr is set up to manage a single “Solr Core” which contains one index. It is also possible to segment Solr into multiple virtual instances of cores, each with its own configuration and indices. Cores can be dedicated to a single application, or to different ones, but all are administered through a common administration interface.
Solr (http://lucene.apache.org/solr/) is the HTTP based server product of the Apache Lucene Project. It uses the Lucene Java library at its core for indexing and search technology, as well as spell checking, hit highlighting, and advanced analysis/ tokenization capabilities.
Multiple Solr Cores can be configured by placing a file named solr.xml in your Solr Home directory, identifying each Solr Core, and the corresponding instance directory for each. When using a single Solr Core, the Solr Home directory is automatically the instance directory for your Solr Core. Configuration of each Solr Core is done through two main config files, both of which are placed in the conf subdirectory for that Core:
www.dzone.com
Michael McCandless Erik Hatcher , Otis Gospodnetic
About Solr Running Solr schema.xml Field Types Analyzers Hot Tips and more...
The fundamental premise of Solr is simple. You feed it a lot of information, then later you can ask it questions and find the piece of information you want. Feeding in information is called indexing or updating. Asking a question is called a querying.
SECOND EDITION
Trey Grainger Timothy Potter
Apache Solr:
CONTENTS INCLUDE:
F OREWORD BY D OUG C UTTING
sschema.xml: where you describe your data s : where you describe how people can interact with your data. By default, Solr will store the index inside the data subdirectory for that Core.
Solr Administration Administration for Solr can be done through http://[hostname]:8983 /solr/admin which provides a section with menu items for monitoring indexing and performance statistics, information about index distribution and replication, and information on all threads running in the JVM at the time. There is also a section where you can run queries, and an assistance area.
Figure 1: A typical Solr setup
Core Solr Concepts
MANNING
MANNING
Apache Solr
Solr’s basic unit of information is a document: a set of information that describes something, like a class in Java. Documents themselves are composed of fields. These are more specific pieces of information, like attributes in a class.
Get the Solr Reference Guide!
RUNNING SOLR
Solr Installation The LucidWorks for Solr installer (http://www.lucidimagination. com/Downloads/LucidWorks-for-Solr) makes it easy to set up your initial Solr instance. The installer brings you through configuration and deployment of the Web service on either Jetty or Tomcat.
Free download at bit.ly/solrguide Check out the Solr and Lucene docs, webcasts, white papers and tech ar!cles at lucidimagina!on.com
Solr Home Directory Solr Home is the main directory where Solr will look for configuration files, data and plug-ins. DZone, Inc.
|
www.dzone.com
MapKit, Solr and RestKit
https://github.com/cjudd/solrmap
Christopher M. Judd CTO and Partner email: [email protected] web: www.juddsolutions.com blog: juddsolutions.blogspot.com twitter: javajudd
Christopher M. Judd
Christopher M. Judd CTO and Partner at leader Developer User Group (CIDUG) BROUGHT TO YOU BY:
C O NT E NT S
Get More Refcardz! Visit dzone.com/refcardz
221
Getting Started With Docker
» About Docker » Docker Architecture » Getting Started » Typical Local Workflow » Other Helpful Commands
By Christopher M. Judd
» Dockerfile, and more...
A B O U T D O CK E R Almost overnight, Docker has become the de facto standard that developers and system administrators use for packaging, deploying, and running distributed applications. It provides tools for simplifying DevOps by enabling developers to create templates called images that can be used to create lightweight virtual machines called containers, which include their applications and all of their applications’ dependencies. These lightweight virtual machines can be promoted through testing and production environments where sysadmins deploy and run them. Docker Images
Docker makes it easier for organizations to automate infrastructure, isolate applications, maintain consistency, and improve resource utilizations.
A recipe or template for creating Docker containers. It includes the steps for installing and running the necessary software.
Docker Like a tiny virtual machine that is created from Container the instructions found within the Docker image originated
Similar to the popular version control software Git, Docker has a social aspect, in that developers and sysadmins are able to share their images via
Docker Client
Command-line utility or other tool that takes advantage of the Docker API (https://docs.docker. com/reference/api/docker_remote_api) to communicate with a Docker daemon
Docker Host
A physical or virtual machine that is running a Docker daemon and contains cached images as well as runnable containers created from images
Docker Hub.
G E T T I N G S TA R T E D W I T H D O C K E R
Columbus
Docker is an open-source solution that runs natively on Linux but also works on Windows and Mac using a lightweight Linux distribution and VirtualBox. Many
Site24x7 DOCKER MONITORING
tools have also grown up around Docker to make it easier to manage and orchestrate complex distributed applications.
Get Detailed Insight into Docker Containers
DO CK ER A RCHITEC TU R E Docker utilizes a client-server architecture and a remote API to manage and create Docker containers built upon
CPU
Docker images. The relationship between containers and images are analogous to the relationship between objects and classes in object-oriented programming.
© DZONE, INC.
Cache
Memory
Linux containers. Docker containers are created from
|
DZONE.COM
I/O
searching is easy right?
select * from products where name = ‘iPhone 5’
select * from products where description like ‘%iphone%’
select * from products where match(name, description) against(‘+iphone -case’ in boolean mode);
http://lucene.apache.org/
Users%
Search'User'Interface' Build& Query&
Render& Result&
Run$Query$
Index&
Index&Document& Analyze(Document(
Build&Document&
Acquire(Document( Raw$ Content$
public class InMemoryExample { public static void main(String[] args) throws CorruptIndexException, LockObtainFailedException, IOException, ParseException { // in-memory representation of the index RAMDirectory idx = new RAMDirectory(); // Make an writer to create the index IndexWriterConfig config = new IndexWriterConfig(LUCENE_36, new StandardAnalyzer(LUCENE_36)); IndexWriter writer = new IndexWriter(idx, config); // Add some Document objects containing quotes writer.addDocument(createDocument( "Theodore Roosevelt", "It behooves every man to remember that the work of the " + "critic, is of altogether secondary importance, and that, " + "in the end, progress is accomplished by the man who does " + "things.")); writer.addDocument(createDocument( "Friedrich Hayek", "The case for individual freedom rests largely on the " + "recognition of the inevitable and universal ignorance " + "of all of us concerning a great many of the factors on " + "which the achievements of our ends and welfare depend.")); writer.addDocument(createDocument( "Ayn Rand", "There is nothing to take a man's freedom away from " + "him, save other men. To be free, a man must be free " + "of his brothers.")); writer.addDocument(createDocument( "Mohandas Gandhi", "" + "Freedom is not worth having if it does not connote " + "freedom to err.")); writer.close(); // Build an IndexSearcher using the in-memory index IndexReader reader = IndexReader.open(idx); IndexSearcher searcher = new IndexSearcher(reader); // Run some queries search(searcher, "freedom"); search(searcher, "free"); search(searcher, "progress or achievements"); searcher.close(); }
search(searcher, "progress or achievements"); searcher.close(); } /** * Make a Document object with an un-indexed title field and an indexed content field. */ private static Document createDocument(String title, String content) { Document doc = new Document(); // Add the title as an unindexed field... doc.add(new Field("title", title, Field.Store.YES, Field.Index.NOT_ANALYZED)); doc.add(new Field("content", content, Field.Store.YES, Field.Index.ANALYZED)); return doc; } /** * Searches for the given string in the "content" field */ private static void search(IndexSearcher searcher, String queryString) throws ParseException, IOException { // Build a Query object Query query = new QueryParser(LUCENE_36, "content", new StandardAnalyzer(LUCENE_36)).parse(queryString); TopDocsCollector collector = TopScoreDocCollector.create(10, true); searcher.search(query, collector); if (collector.getTotalHits() == 0) { System.out.println("No matches were found for \"" + queryString + "\""); } else { System.out.println("Hits for \"" + queryString + "\" were found in quotes by:"); ScoreDoc[] hits = collector.topDocs().scoreDocs; for (ScoreDoc hit : hits) { Document doc = searcher.doc(hit.doc); System.out.println(" - " + doc.get("title")); } } System.out.println(); } }
search(searcher, "progress or achievements"); searcher.close(); } /** * Make a Document object with an un-indexed title field and an indexed content field. */ private static Document createDocument(String title, String content) { Document doc = new Document(); // Add the title as an unindexed field... doc.add(new Field("title", title, Field.Store.YES, Field.Index.NOT_ANALYZED)); doc.add(new Field("content", content, Field.Store.YES, Field.Index.ANALYZED)); return doc; } /** * Searches for the given string in the "content" field */ private static void search(IndexSearcher searcher, String queryString) throws ParseException, IOException { // Build a Query object Query query = new QueryParser(LUCENE_36, "content", new StandardAnalyzer(LUCENE_36)).parse(queryString); TopDocsCollector collector = TopScoreDocCollector.create(10, true); searcher.search(query, collector); if (collector.getTotalHits() == 0) {Hits for "freedom" were found in quotes System.out.println("No matches were found for \"" + queryString + "\""); - Mohandas Gandhi } else { System.out.println("Hits for \"" + queryString Ayn Rand + "\" were found in quotes by:");
by:
- Friedrich Hayek
ScoreDoc[] hits = collector.topDocs().scoreDocs; for (ScoreDoc hit : hits) { Document doc = searcher.doc(hit.doc); Hits for "free" System.out.println(" - " + doc.get("title")); - Ayn Rand } } System.out.println(); } }
were found in quotes by:
Hits for "progress or achievements" were found in quotes by: - Theodore Roosevelt - Friedrich Hayek
Index Document Field Value title
Field
Term
Value
Term
Document Field Value
Term
content
title
Field content
…
Value
Term
Theodore Roosevelt
It
Term
behooves
Term
every
…
case
Term
for
…
Fredrich Hayek
The
Term
Documents 1: Theodore Roosevelt It behooves every man to remember that the work of the critic, is of altogether secondary importance, and that, in the end, progress…
Term Index
… progress
…
1
achievements
1
free
3
freedom
… …
2
2: Friedrich Hayek
2
3
The case for individual freedom rests largely on the recognition of the inevitable and universal ignorance of all of us concerning …
3: Ayn Rand There is nothing to take a man's freedom away from him, save other men. To be free, a man must be free of his brothers.
popular powerful
Java only single threaded writes lots of boiler plate code difficult to debug and test indexes no user interface
http://lucene.apache.org/solr/
scalable caching highlighting boosting facets clustering database integration rich document (Word, PDF) indexing geospatial search REST-like HTTP/XML and JSON API
Application
datasources
bin/solr start -e cloud Welcome to the SolrCloud example! To begin, how many Solr nodes would you like to run in your local cluster? (specify 1-4 nodes) [2] Please enter the port for node1 [8983] Please enter the port for node2 [7574] Starting up SolrCloud node1 on port 8983 using command: solr start -cloud -s example/cloud/node1/solr -p 8983
-m 512m
Waiting to see Solr listening on port 8983 [/] Started Solr server on port 8983 (pid=31995). Happy searching! Starting node2 on port 7574 using command: solr start -cloud -s example/cloud/node2/solr -p 7574 -z localhost:9983 -m 512m Waiting to see Solr listening on port 7574 [/] Started Solr server on port 7574 (pid=32086). Happy searching! Now let's create a new collection for indexing documents in your 2-node cluster. Please provide a name for your new collection: [gettingstarted] workshop workshop How many shards would you like to split workshop into? [2] 2 How many replicas per shard would you like to create? [2] 2 Please choose a configuration for the workshop collection, available options are: basic_configs, data_driven_schema_configs, or sample_techproducts_configs [data_driven_schema_configs] sample_techproducts_configs Connecting to ZooKeeper at localhost:9983
Creating new collection 'workshop' using command: http://172.25.5.134:7574/solr/admin/collections?action=CREATE&name=workshop&numShards=2&replicationFactor=2&maxShardsPerNode=2&collection.con {“responseHeader”:{“status”:0,"QTime":4017}, “success”:{“”:{"responseHeader":{"status":0,"QTime":3807},"core":"workshop_shard2_replica1"}}} Enabling auto soft-commits with maxTime 3 secs using the Config API POSTing request to Config API: http://localhost:8983/solr/workshop/config {"set-property":{"updateHandler.autoSoftCommit.maxTime":"3000"}} Successfully set-property updateHandler.autoSoftCommit.maxTime to 3000 SolrCloud example running, please visit http://localhost:8983/solr
http://localhost:8983/solr/
Stop bin/solr stop -all Restart bin/solr start -e cloud -noprompt
Setup Lab
1. Install Solr >= 5.2.1 2. Start Solr 3. Browse to http://localhost:8983/solr/
Basic Querying
Users%
Search'User'Interface' Build& Query&
Render& Result&
Run$Query$
Index&
request handler
output format
http://localhost:8983/solr/gettingstarted_shard1_replica1/select?q=*%3A*&wt=json
index
query for everything
{"responseHeader":{"status":0,"QTime":14,"params": {"q":"*:*","wt":"json"}},"response":{"numFound":0,"start": 0,"maxScore":0.0,"docs":[]}}
/select?q=*:*&wt=json&indent=on pretty print
{ "responseHeader":{ "status":0, "QTime":4, "params":{ "q":"*:*", "indent":"true", "wt":"json"}}, "response":{"numFound":0,"start":0,"maxScore":0.0,"docs":[] }}
/select?q=*:*&wt=xml&indent=on xml
0 6 <str name="q">*:* <str name="wt">xml
Index Data
Index&
Index&Document& Analyze(Document(
Build&Document&
Acquire(Document( Raw$ Content$
JSON XML CSV Database
https://developers.google.com/books/docs/v1/getting_started
<doc> false name="description">Written with a rare combination of analysis and
…
[ { "id": "ka2VUBqHiWkC", "title": "Effective Java", "author": "Joshua Bloch", "author_txt": [ "Joshua Bloch" ], "page_i": 368, "saleable_b": true, "description": "Are you looking for a deeper understanding of the Java …”, "price": 25.49 }, { "id": "-SYM4PW-YAgC", "title": "The Religion of Java", "author": "Clifford Geertz", "author_txt": [ "Clifford Geertz" ], "page_i": 392, "saleable_b": false, "description": "Written with a rare combination of analysis and speculation …” } ]
java -Dc=workshop -Dtype=application/json -jar post.jar books/googlebooks*.json
SimplePostTool version 5.0.0 Posting files to [base] url http://localhost:8983/solr/workshop/update using contenttype application/json... POSTing file googlebooks.json to [base] POSTing file googlebooks1.json to [base] POSTing file googlebooks2.json to [base] 3 files indexed. COMMITting Solr index changes to http://localhost:8983/solr/workshop/update... Time spent: 0:00:00.175
Index Lab
1. 2. 3. 4. 5. 6.
Load Google book data Perform a search to determine how many books Return a result in XML Return a human readable result in XML Return a result in JSON Return a human readable result in JSON
More Querying
/select?q=*:*&wt=json&indent=true { "responseHeader": { "status": 0, "QTime": 0, "params": { "indent": "true", "q": "*:*", "_": "1403710200465", "wt": "json" } }, "response": { "numFound": 1, "start": 0, "docs": [ { "id": "ka2VUBqHiWkC", "title": [ "Effective Java" ], "author": "Joshua Bloch", "author_s": "Joshua Bloch", "author_txt": [ "Joshua Bloch" ], "saleable_b": true, "description": "Are you looking for a deeper understanding of the Java ...", "_version_": 1471896715487346700 } ] } }
Limit Fields /select?q=*:*&fl=name id&wt=json&indent=true {
"responseHeader":{ "status":0, "QTime":0, "params":{ "fl":"name id", "indent":"true", "q":"*:*", "wt":"json"}}, "response":{"numFound":10,"start":0,"docs":[ { "id":"ka2VUBqHiWkC", "name":"Effective Java"}, { "id":"-SYM4PW-YAgC", "name":"The Religion of Java"}, { "id":"mB_92VqJbsMC", "name":"Java Threads"}, { "id":"Ql6QgWf6i7cC", "name":"Thinking in Java"}, { "id":"zuGy-V3Nk4AC", "name":"Java SE 7 Programming Essentials"}, { "id":"mvzgNSmHEUAC", "name":"Java in a Nutshell"}, { "id":"gJEC2q7DzpQC", "name":"The History of Java"}, { "id":"vvg7fN_HScAC", "name":"Advanced Java Networking"}, { "id":"pnwTLvCJKh0C", "name":"Java Programming"}, { "id":"Y0lDBsh7J9kC", "name":"Learning Java"}] }}
Limit Results /select?q=*:*&rows=5&fl=name+id&wt=json&indent=true
{
"responseHeader":{ "status":0, "QTime":0, "params":{ "fl":"name id", "indent":"true", "q":"*:*", "wt":"json", "rows":"5"}}, "response":{"numFound":10,"start":0,"docs":[ { "id":"ka2VUBqHiWkC", "name":"Effective Java"}, { "id":"-SYM4PW-YAgC", "name":"The Religion of Java"}, { "id":"mB_92VqJbsMC", "name":"Java Threads"}, { "id":"Ql6QgWf6i7cC", "name":"Thinking in Java"}, { "id":"zuGy-V3Nk4AC", "name":"Java SE 7 Programming Essentials"}] }}
Pagination /select?q=*:*&start=5&rows=5&fl=name+id&wt=json&indent=true
{
"responseHeader":{ "status":0, "QTime":0, "params":{ "fl":"name id", "indent":"true", "start":"5", "q":"*:*", "wt":"json", "rows":"5"}}, "response":{"numFound":10,"start":5,"docs":[ { "id":"mvzgNSmHEUAC", "name":"Java in a Nutshell"}, { "id":"gJEC2q7DzpQC", "name":"The History of Java"}, { "id":"vvg7fN_HScAC", "name":"Advanced Java Networking"}, { "id":"pnwTLvCJKh0C", "name":"Java Programming"}, { "id":"Y0lDBsh7J9kC", "name":"Learning Java"}] }}
Query Field /select?q=name:Java&fl=name+id&wt=json&indent=true {
"responseHeader":{ "status":0, "QTime":8, "params":{ "q":"name_txt:java", "indent":"true", "fl":"name id", "wt":"json"}}, "response":{"numFound":10,"start":0,"maxScore":0.51104903,"docs":[ { "id":"mB_92VqJbsMC", "name":["Java Threads"]}, { "id":"ka2VUBqHiWkC", "name":["Effective Java"]}, { "id":"pnwTLvCJKh0C", "name":["Java Programming"]}, { "id":"Y0lDBsh7J9kC", "name":["Learning Java"]}, { "id":"-SYM4PW-YAgC", "name":["The Religion of Java"]}, { "id":"mvzgNSmHEUAC", "name":["Java in a Nutshell"]}, { "id":"gJEC2q7DzpQC", "name":["The History of Java"]}, { "id":"vvg7fN_HScAC", "name":["Advanced Java Networking"]}, { "id":"Ql6QgWf6i7cC", "name":["Thinking in Java"]}, { "id":"zuGy-V3Nk4AC", "name":["Java SE 7 Programming Essentials"]}] }}
Query Field /select?q=author:Bloch&wt=json&indent=true {
"responseHeader":{ "status":0, "QTime":9, "params":{ "q":"author:Bloch", "indent":"true", "wt":"json"}}, "response":{"numFound":1,"start":0,"maxScore":1.1976817,"docs":[ { "id":"ka2VUBqHiWkC", "name":"Effective Java", "author":"Joshua Bloch", "author_s":"Joshua Bloch", "author_txt":["Joshua Bloch"], "pages_i":368, "saleable_b":true, "description":"Are you looking for a deeper understanding of the Java™ programming language so that you can write code tha is clearer, more correct, more robust, and more reusable? Look no further! Effective Java™, Second Edition, brings together seventy-eight indispensable programmer’s rules of thumb: working, best-practice solutions for the programming challenges you encounter every day. This highly anticipated new edition of the classic, Jolt Award-winning work has been thoroughly updated to cover Java SE 5 and Java SE 6 features introduced since the first edition. Bloch explores new design patterns and language idioms, showing you how to make the most of features ranging from generics to enums, annotations to autoboxing. Each chapter in the book consists of several “items” presented in the form of a short, standalone essay that provides specific advice, insight into Java platform subtleties, and outstanding code examples. The comprehensive descriptions and explanations for each item illuminate what to do, what not to do, and why. Highlights include: New coverage of generics, enums, annotations, autoboxing, th for-each loop, varargs, concurrency utilities, and much more Updated techniques and best practices on classic topics, including objects, classes, libraries, methods, and serialization How to avoid the traps and pitfalls of commonly misunderstood subtleties of the language Focus on the language and its most fundamental libraries: java.lang, java.util, and, to a lesser extent, java.util.concurrent and java.io Simply put, Effective Java™, Second Edition, presents the most practical, authoritative guidelines available for writing efficient, well-designed programs.", "cat":["Computers"], "price":25.49, "price_c":"25.49,USD", "_version_":1507586189982433280}] }}
Query Field /select?q=author_s:Bloch&wt=json&indent=true
{ "responseHeader": { "status": 0, "QTime": 17, "params": { "q": "author_s:Bloch", "indent": "true", "wt": "json", "_": "1437746491580" } }, "response": { "numFound": 0, "start": 0, "maxScore": 0, "docs": [] } }
Query Multiple Fields /select?q=name:Java+AND+saleable_b:true &fl=name+id+saleable_b&wt=json&indent=true
{ "responseHeader":{ "status":0, "QTime":9, "params":{ "q":"name:java AND saleable_b:true", "indent":"true", "fl":"name id saleable_b", "wt":"json"}}, "response":{"numFound":6,"start":0,"maxScore":1.3008728,"docs":[ { "id":"mB_92VqJbsMC", "name":["Java Threads"], "saleable_b":true}, { "id":"ka2VUBqHiWkC", "name":["Effective Java"], "saleable_b":true}, { "id":"Y0lDBsh7J9kC", "name":["Learning Java"], "saleable_b":true}, { "id":"mvzgNSmHEUAC", "name":["Java in a Nutshell"], "saleable_b":true}, { "id":"gJEC2q7DzpQC", "name":["The History of Java"], "saleable_b":true}, { "id":"zuGy-V3Nk4AC", "name":["Java SE 7 Programming Essentials"], "saleable_b":true}] }}
Query Ranges /select?q=pages_i:[*+TO+400] &fl=name+id+pages_i&wt=json&indent=true
{
"responseHeader":{ "status":0, "QTime":0, "params":{ "fl":"name id pages_i", "indent":"true", "q":"pages_i:[* TO 400]", "wt":"json"}}, "response":{"numFound":5,"start":0,"docs":[ { "id":"ka2VUBqHiWkC", "name":"Effective Java", "pages_i":368}, { "id":"-SYM4PW-YAgC", "name":"The Religion of Java", "pages_i":392}, { "id":"mB_92VqJbsMC", "name":"Java Threads", "pages_i":340}, { "id":"zuGy-V3Nk4AC", "name":"Java SE 7 Programming Essentials", "pages_i":336}, { "id":"vvg7fN_HScAC", "name":"Advanced Java Networking", "pages_i":399}] }}
Sorting /select?q=pages_i:[*+TO+400]&sort=pages_i+asc &fl=name+id+pages_i&wt=json&indent=true
{
"responseHeader":{ "status":0, "QTime":0, "params":{ "fl":"name id pages_i", "sort":"pages_i asc", "indent":"true", "q":"pages_i:[* TO 400]", "wt":"json"}}, "response":{"numFound":5,"start":0,"docs":[ { "id":"zuGy-V3Nk4AC", "name":"Java SE 7 Programming Essentials", "pages_i":336}, { "id":"mB_92VqJbsMC", "name":"Java Threads", "pages_i":340}, { "id":"ka2VUBqHiWkC", "name":"Effective Java", "pages_i":368}, { "id":"-SYM4PW-YAgC", "name":"The Religion of Java", "pages_i":392}, { "id":"vvg7fN_HScAC", "name":"Advanced Java Networking", "pages_i":399}] }}
Facets
Facets /select?q=*:*&rows=0&wt=json&indent=true &facet=true&facet.field=cat { "responseHeader":{ "status":0, "QTime":35, "params":{ "q":"*:*", "facet.field":"cat", "indent":"true", "rows":"0", "wt":"json", "facet":"true"}}, "response":{"numFound":10,"start":0,"maxScore":1.0,"docs":[] }, "facet_counts":{ "facet_queries":{}, "facet_fields":{ "cat":[ "Computers",8, "Java (Indonesia)",1, "Religion",1]}, "facet_dates":{}, "facet_ranges":{}, "facet_intervals":{}, “facet_heatmaps”:{} } }
Highlighting /select?q=description:java AND name:"Java in a Nutshell"&wt=json&indent=true &hl=true &hl.fl=description &hl.simple.pre=<em> &hl.simple.post=
{
}
"responseHeader":{ "status":0, "QTime":28, "params":{ "indent":"true", "q":"description:java AND title:\"Java in a Nutshell\" ", "hl.simple.pre":"<em>", "hl.simple.post":"<em>", "hl.fl":"description", "wt":"json", "hl":"true"}}, "response":{"numFound":1,"start":0,"docs":[ { "id":"mvzgNSmHEUAC", "title":["Java in a Nutshell"], "author":"David Flanagan", "author_s":"David Flanagan", "author_txt":["David Flanagan"], "page_i":1224, "saleable_b":true, "description":"Aimed for programmers, offers an introduction to Java 5.0 ...", "_version_":1471896715499929600}] }, "highlighting":{ "mvzgNSmHEUAC":{ "description":["Aimed for programmers, offers an introduction to <em>Java 5.0, covering topics such as generics”]} }
Query Lab
1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
Search for all books but only return the id, name and author Return a result with all books Search for any books with java Search for any books without java Search for any books with java in the title (name) Search for any books with java in the title and are available to purchase Search for books between 100 and 200 pages Sort all the books based on the author's name Determine how many books there in each category Highlight the word java in the description of all the books
Configurations
<solr_home>/server/solr/configsets/sample_techproducts_configs/conf
schema.xml
<schema>
stored="true"/> stored="true" multiValued="true"/> stored="true"/> stored="true"/>
id
Users%
Search'User'Interface' Build& Query&
Render& Result&
Run$Query$
Index&
Index&Document& Analyze(Document(
Build&Document&
Acquire(Document( Raw$ Content$
Solr
Query
Response
Request Handler
Response Writer
Lucene index Update Handler
Data Source
solrconfig.xml
<requestHandler name="/select" > <str name="echoParams">explicit 10 <str name="df">text
<requestHandler name="/query" > <str name="echoParams">explicit <str name="wt">json <str name="indent">true <str name="df">text
<requestHandler name="/browse" > <str name="echoParams">explicit
VelocityResponseWriter settings --> name="wt">velocity name="v.template">browse name="v.layout">layout name="title">Solritas
<arr name="last-components"> <str>spellcheck
Configuration Lab
1. 2. 3. 4. 5. 6.
Add a new title field to the schema. Make the title searchable and sortable Restart Solr Reindex Search for Java in the titles Sort titles containing Java in them
http://localhost:8983/solr/workshop/update? stream.body=<delete>*:*
Rich Content
bin/post -c workshop ../solr-data/refcardz/ http://localhost:8983/solr/workshop/select? q=docker&rows=200&wt=json&indent=true { "responseHeader":{ "status":0, "QTime":50, "params":{ "q":"docker", "indent":"true", "wt":"json"}}, "response":{"numFound":1,"start":0,"maxScore":0.05551956,"docs":[ { "links":[ "https://github.com/cloudfoundry", "http://www.refcardz.com", "http://cloudfoundry.org/",], "id":"/Users/cjudd/devl/tmp/solr-5.2.1/../solr-data/refcardz/rc207_010d-cloudfoundry_0.pdf", "resourcename":"/Users/cjudd/devl/tmp/solr-5.2.1/../solr-data/refcardz/rc207_010d-cloudfoundry_0.pdf", "last_modified":"2015-04-15T19:02:41Z", "content_type":["application/pdf"], "content":["\n\n Get Started »\nLearn more: stackato.com/refcard\n \n Deploy apps faster on any cloud with Stackato, the leading\nenterprise PaaS based on Cloud Foundry, Docker and other open\nsource technologies. Free to use in production up to 20GB.\n \n The Easiest Way to \nGet Started with \nCloud Foundry \n \n \n \n \n © DZone, Inc. | DZone.com\n \n cloud Foundry\nopen-Source PaaS for Streamlined Development and Deployment\n \n By Jeremy Voorhis, Revised and Updated by Billy Tat\n \n » Supported Technologies \n \n » Your "], "_version_":1507640053787000832}] }}
bin/post -c workshop http://www.cojug.org -recursive 1 -delay 1
http://localhost:8983/solr/workshop/select? q=*:*&rows=1&wt=json&indent=true {
"responseHeader":{ "status":0, "QTime":31, "params":{ "q":"*:*", "indent":"true", "rows":"1", "wt":"json"}}, "response":{"numFound":23,"start":0,"maxScore":1.0,"docs":[ { "links":["rect", "mainlevel-nav", "http://www.cojug.org/index.php",], "id":"http://www.cojug.org", "url":"http://www.cojug.org", "keywords":"COJUG, Java, Ohio, User Group", "description":"COJUG - Central Ohio Java User's Group", "title":["Central Ohio Java Users Group (COJUG) - Home"], "content_type":["text/html; charset=iso-8859-1"], "content":["\n Central Ohio Java Users Group (COJUG) - Home \n \n Saturday, 25 July 2015\n\nHome\tContact Us\n\tNews ... pageTracker._initData(); \npageTracker._trackPageview();\n \n\n "], "_version_":1507640692579500032}] }}
Geospatial
http://poiplaza.com/
java -Dc=workshop -Dtype=application/json -jar post.jar parks/ amusement_parks.json
[
{
}, {
"id": "Myrtle Waves Water Park", "name": "Myrtle Waves Water Park", "description": "USA-North Myrtle Beach
\nUS 17 Bypass at 10th Avenue
\n", "coordinates_p": "33.81673,-78.68005", "store": "33.81673,-78.68005", "source_s": "USA-Amusement & Theme Parks"
"id": "Coney Island", "name": "Coney Island", "description": "USA-New York-Brooklyn
\n1208 Surf Ave.
\n+1 718-372-5159", "coordinates_p": "40.57546,-73.98017", "store": "40.57546,-73.98017", "source_s": "USA-Amusement & Theme Parks" }, { "id": "Six Flags Over Georgia", "name": "Six Flags Over Georgia", "description": "USA-Austell
\n7561 Six Flags Pkwy
\n+1 770-948-9290", "coordinates_p": "33.77091,-84.55220", "store": "33.77091,-84.55220", "source_s": "USA-Amusement & Theme Parks" } ]
http://localhost:8983/solr/parks/browse
distance filters point distance bounding lat and long
point distance /select?q=*:*&fq={!geofilt}&pt=37.7752,-122.4232&d=100&sfield=coordinates_p&wt=json
point distance filter
{
km
"id": "Myrtle Waves Water Park", "name": "Myrtle Waves Water Park", "description": "USA-North Myrtle Beach< "coordinates_p": "33.81673,-78.68005", "store": "33.81673,-78.68005", "source_s": "USA-Amusement & Theme Park }
bounding latitude and longitude /select?q=*:*&fq={!bbox}&pt=37.7752,-122.4232&d=100&sfield=coordinates_p&wt=json
bounding filter
{
km
"id": "Myrtle Waves Water Park", "name": "Myrtle Waves Water Park", "description": "USA-North Myrtle Beach< "coordinates_p": "33.81673,-78.68005", "store": "33.81673,-78.68005", "source_s": "USA-Amusement & Theme Park }
calculate distance /select?q=*:*&fq={!bbox} &pt=37.7752,-122.4232&d=100&sfield=coordinates_p&wt=json&fl=_dist_:geodist(),name,de scription,coordinates_p&indent=on {
"responseHeader":{ "status":0, "QTime":0, "params":{ "d":"100", "fl":"_dist_:geodist(),name,description,coordinates_p", "indent":"on", "q":"*:*", "sfield":"coordinates_p", "pt":"37.7752,-122.4232", "wt":"json", "fq":"{!bbox}"}}, "response":{"numFound":2,"start":0,"docs":[ { "name":"Six Flags Marine World", "description":"USA-Vallejo
\n2001 Marine World Parkway
\n+1 707-643-6722", "coordinates_p":"38.14176,-122.24957", "_dist_":43.50948310887128}, { "name":"Great America", "description":"USA-Santa Clara
\n2401 Agnew Rd
\n", "coordinates_p":"37.39057,-121.96905", "_dist_":58.57220580622602}] }}
/select?q=*:*&fq={!bbox} &pt=37.7752,-122.4232&d=1000&sfield=coordinates_p&wt=json&fl=_dist_:geodist(),name,d escription,coordinates_p&indent=on&sort=geodist()+asc {
sort by distance
"responseHeader":{ "status":0, "QTime":14, "params":{"d":"1000", "fl":"_dist_:geodist(),name,description,coordinates_p", "sort":"geodist() asc", "indent":"on","q":"*:*","sfield":"coordinates_p","pt":"37.7752,-122.4232", "wt":"json", "fq":"{!bbox}"}}, "response":{"numFound":23,"start":0,"docs":[ { "name":"Six Flags Marine World", "description":"USA-Vallejo
\n2001 Marine World Parkway
\n+1 707-643-6722", "coordinates_p":"38.14176,-122.24957", "_dist_":43.50948310887128}, { "name":"Great America", "description":"USA-Santa Clara
\n2401 Agnew Rd
\n", "coordinates_p":"37.39057,-121.96905", "_dist_":58.57220580622602}, { "name":"Universal Studios Hollywood", "description":"USA-Universal City
\n100 Universal City Plaza
\n+1 800-959-9688", "coordinates_p":"34.13673,-118.35590", "_dist_":545.5031109696113}, { "name":"Santa Monica Pier", "description":"USA-Santa Monica
\n200 Santa Monica Pier
\n+1 310-458-8900", "coordinates_p":"34.01036,-118.49612", "_dist_":547.9619130243483}, { "name":"Raging Waters", "description":"USA-San Dimas
\n111 Raging Waters Drive
\n+1 909-802-2200", "coordinates_p":"34.08565,-117.81186", "_dist_":583.5368017552456}, { "name":"Knott's Berry Farm/Soak City", "description":"USA-Buena Park
\n8039 Beach Blvd.
\n+1 714-220-5200", "coordinates_p":"33.84550,-117.99810", "_dist_":591.5876572669723}, { "name":"Disneyland", "description":"USA-Anaheim
\n700 W Ball Rd
\n+1 714-781-4565", "coordinates_p":"33.81786,-117.91846", "_dist_":598.7499877946933}, {
Geospatial Lab
1. 2. 3.
Clear the current index Load the amusement park data Search for amusement parks near (100 meters) of Orlando Florida (28.405080, -81.579433)
http://www.elasticsearch.org/
ELK collection and processing input | codecs | filters | outputs
indexing and searching
user interface
brought to you by...
Covers Apache Lucene 3.0
IN ACTION
Get More Refcardz! Visit refcardz.com
#120
FOREWORD BY Yonik Seeley
Getting Optimal Search Results
By Chris Hostetter
When LucidWorks is installed at ~/LucidWorks the Solr Home directory is ~/LucidWorks/lucidworks/solr/.
ABOUT SOLR
Single Core and Multicore Setup
Solr makes it easy for programmers to develop sophisticated, high performance search applications with advanced features such as faceting, dynamic clustering, database integration and rich document handling.
By default, Solr is set up to manage a single “Solr Core” which contains one index. It is also possible to segment Solr into multiple virtual instances of cores, each with its own configuration and indices. Cores can be dedicated to a single application, or to different ones, but all are administered through a common administration interface.
Solr (http://lucene.apache.org/solr/) is the HTTP based server product of the Apache Lucene Project. It uses the Lucene Java library at its core for indexing and search technology, as well as spell checking, hit highlighting, and advanced analysis/ tokenization capabilities.
Multiple Solr Cores can be configured by placing a file named solr.xml in your Solr Home directory, identifying each Solr Core, and the corresponding instance directory for each. When using a single Solr Core, the Solr Home directory is automatically the instance directory for your Solr Core. Configuration of each Solr Core is done through two main config files, both of which are placed in the conf subdirectory for that Core:
www.dzone.com
Michael McCandless Erik Hatcher , Otis Gospodnetic
About Solr Running Solr schema.xml Field Types Analyzers Hot Tips and more...
The fundamental premise of Solr is simple. You feed it a lot of information, then later you can ask it questions and find the piece of information you want. Feeding in information is called indexing or updating. Asking a question is called a querying.
SECOND EDITION
Trey Grainger Timothy Potter
Apache Solr:
CONTENTS INCLUDE:
F OREWORD BY D OUG C UTTING
sschema.xml: where you describe your data s : where you describe how people can interact with your data. By default, Solr will store the index inside the data subdirectory for that Core.
Solr Administration Administration for Solr can be done through http://[hostname]:8983 /solr/admin which provides a section with menu items for monitoring indexing and performance statistics, information about index distribution and replication, and information on all threads running in the JVM at the time. There is also a section where you can run queries, and an assistance area.
Figure 1: A typical Solr setup
Core Solr Concepts
MANNING
MANNING
Apache Solr
Solr’s basic unit of information is a document: a set of information that describes something, like a class in Java. Documents themselves are composed of fields. These are more specific pieces of information, like attributes in a class.
Get the Solr Reference Guide!
RUNNING SOLR
Solr Installation The LucidWorks for Solr installer (http://www.lucidimagination. com/Downloads/LucidWorks-for-Solr) makes it easy to set up your initial Solr instance. The installer brings you through configuration and deployment of the Web service on either Jetty or Tomcat.
Free download at bit.ly/solrguide Check out the Solr and Lucene docs, webcasts, white papers and tech ar!cles at lucidimagina!on.com
Solr Home Directory Solr Home is the main directory where Solr will look for configuration files, data and plug-ins. DZone, Inc.
|
www.dzone.com
MapKit, Solr and RestKit
https://github.com/cjudd/solrmap
Christopher M. Judd CTO and Partner email: [email protected] web: www.juddsolutions.com blog: juddsolutions.blogspot.com twitter: javajudd
