1. Building Intelligent Search Applications with Apache Solr and PHP5

2. About the Presenter Israel Ekpo
Software Architect with Bonnier Corporation
Author of Apache Solr PECL extension
Website:

3. Why Search? Looking for needle in haystack.
Retrieve information quickly.
Retrieve information and relevant results.
Narrow down result sets.
Sell more products to customers.
Display content of interest to the visitors.
Keep users staying on the web applications.
Increase the number of returning users.

4. How to Implement Search
MySQL (full text search with MyISAM)‏
Sphinx
Lucene
Apache Solr

5. Search Tool of Choice?
Apache Solr

6. Apache Solr Features HTTP interface for clients (any language)‏
Standalone powerful full-text search server
REST-like HTTP/XML and JSON APIs
Hit highlighting
Faceted search
Dynamic clustering
Database integration
Open Source (FREE)‏

7. Where do I get Solr? http://www.apache.org/dyn/closer.cgi/lucene/solr/
Current Version is 1.4.0

8. How do I Install It or Set it Up?
Solr can run as a Standalone Search Server
However for production purposes it is recommended to set it up with a servelet container or application server such as Jetty, Tomcat or Glassfish.

9. Setting Up Tomcat to Work with Solr
Verifying Availability of Java 1.6 Dependencies
$ dpkg --get-selections | grep sun-java
sun-java6-bin install
sun-java6-jdk install
sun-java6-jre install
$ sudo aptitude install sun-java6-jdk sun-java6-bin sun-java6-jre

10. Getting Tomcat 6 http://tomcat.apache.org/download-60.cgi
Get the URL to the Core Binary distribution from your closest mirror and then
$ wget 6/v6.0.20/bin/apache-tomcat tar.gz
$ tar -zxvf apache-tomcat tar.gz

11. Setting up Tomcat 6 $ sudo mv apache-tomcat-6.0.20 /usr/local/tomcat
The $JAVA_HOME and $JAVA_OPTS variables are required so we have to declare these in the ~/.bashrc file.
The $JAVA_HOME variable will be set to /usr/lib/jvm/java-6-sun
The Solr home will be set to /usr/local/tomcat/solr

12. Contents of .bashrc file
export JAVA_HOME=/usr/lib/jvm/java-6-sun
export JAVA_OPTS="$JAVA_OPTS - Dsolr.solr.home=/usr/local/tomcat/solr - Dsolr.data.dir=/usr/local/tomcat/solr/data - Dsolr.abortOnConfigurationError=true"
For multi-core configuration, please remove - Dsolr.data.dir=/usr/local/tomcat/solr/data from the options

13. Tomcat Manual Setup Complete
The set up for Tomcat is now complete.
It can be started and stopped using the following commands :
$ sudo /usr/local/tomcat/bin/startup.sh
$ sudo /usr/local/tomcat/bin/shutdown.sh

14. Setting up Tomcat Admin and Users
This will be done in the $CATALINA_HOME/conf/tomcat-users.xml file
$ sudo vim /usr/local/tomcat/conf/tomcat-users.xml
<tomcat-users>
<role rolename="manager"/>
<role rolename="admin"/>
<role rolename="webuser"/>
<user username="admin" password="Ch8ng3me" roles="admin,manager,webuser"/>
<user username="frontend" password="Ch8ng3me" roles="webuser"/>
</tomcat-users>

15. Port number for HTTP Connector
The default port for the Java HTTP Connector is 8080
This should be the first /Server/Service/Connector element node.
The second /Server/Connector node is for the Java AJP Connector.
$ sudo vim /usr/local/tomcat/conf/server.xml
<Server ...>
<Service ...>
<Connector port="8983" ... />
...
</Connector>
</Service>
</Server>

16. Character Encoding for Non-ASCII
Find the node /Server/Service/Connector element node and add or set its
"URIEncoding" attribute to "UTF-8".
$ sudo vim /usr/local/tomcat/conf/server.xml
<Server ...>
<Service ...>
<Connector ... URIEncoding="UTF-8"/>
...
</Connector>
</Service>
</Server>

17. Automatic Startups and Shutdowns
Create the file /etc/init.d/tomcat and enter startup and shutdown commands :
$ sudo vim /etc/init.d/tomcat

18. Automatic Shutdown and Startups
case $1 in
start)‏
sh /usr/local/tomcat/bin/startup.sh ;;
stop)
sh /usr/local/tomcat/bin/shutdown.sh
restart)‏
esac
exit 0

19. Automatic Shutdown and Startups
We have to make the /etc/init.d/tomcat script executable :
$ sudo chmod 0755 /etc/init.d/tomcat
The final step is to create a symbolic link between the /etc/init.d/tomcat script to the startup and shutdown folders.
$ sudo ln -s /etc/init.d/tomcat /etc/rc1.d/K99tomcat
$ sudo ln -s /etc/init.d/tomcat /etc/rc2.d/S99tomcat
Terminamos!

20. Setting up Apache Solr 1. Download Solr 1.4.0
$ wget
2. Unzip the compressed folder
$ unzip apache-solr zip
3. Copy the solr.war file to the Tomcat webapps directory :
$ sudo cp -p apache-solr-1.4.0/example/webapps/solr.war /usr/local/tomcat/webapps/solr.war

21. Setting Up Solr 4. We now have to set up the Solr Home.
Copy the example solr home example/solr as a template for your solr home.
$ sudo cp -pr apache-solr-1.4.0/example/solr /usr/local/tomcat/solr
From here on $SOLR_HOME is /usr/local/tomcat/solr
The default solrconfig.xml file in $SOLR_HOME/conf/solrconfig.xml set the data directory
for the index as ./solr/data relative to the current working directory.
Please modify this to the absolute path to $SOLR_HOME/data
$sudo vim /usr/local/tomcat/solr/solrconfig.xml
<dataDir>${solr.data.dir:/usr/local/tomcat/solr/data}</dataDir>

22. Setting Up Solr 5. We are almost done.
We now have to restart the servlet container
$ sudo /etc/init.d/tomcat restart

23. Setting Up Solr
6. If you are setting up Solr in multi-core mode, then you need to set up the solr.xml in the $SOLR_HOME folder
The core name, instance and data directory will be specified in the solr.xml file, each one with its own schema.xml and solrconfig.xml files
This can be accomplished by simply moving the original conf folder to the instance directory for each core and then changing the contents of the files to match your settings

24. Running Solr in Multi-Core Mode
<?xml version='1.0' encoding='UTF-8'?>
<solr persistent="true" sharedLib="lib">
<cores adminPath="/admin/cores" shareSchema="false">
<core name="confooevents" instanceDir="confooevents">
<property name="dataDir" value="/usr/local/tomcat/solr/data/confooevents" />
</core>
<core name="confoospeakers" instanceDir="confoospeakers" >
<property name="dataDir" value="/usr/local/tomcat/solr/data/confoospeakers" />
<core name="confoosuggest" instanceDir="confoosuggest" >
<property name="dataDir" value="/usr/local/tomcat/solr/data/confoosuggest" />
</cores>
</solr>

25. schema.xml <fields>
<field name="speaker_id" type="tint" indexed="true" stored="true" required="true" />
<field name="speaker_name" type="string" indexed="true" stored="true" multiValued="false" omitNorms="true"/>
<field name="company" type="string" indexed="true" stored="true" multiValued="false" omitNorms="true"/>
<field name="talks" type="string" indexed="true" stored="true" multiValued="true" omitNorms="true"/>
<field name="number_talks" type="tint" indexed="true" stored="true" multiValued="false" omitNorms="true"/>
<field name="bio" type="text" indexed="true" stored="true" multiValued="false"/>
<field name="text" type="text" indexed="true" stored="false" multiValued="true"/>
<!-- Copying from display to default search fields. see text above and defaultSearchField below -->
<copyField source="speaker_name" dest="text" />
<copyField source="company" dest="text" />
<copyField source="talks" dest="text" />
<copyField source="bio" dest="text" />
</fields>
<!-- This is the primary key for this index -->
<uniqueKey>speaker_id</uniqueKey>

26. solrconfig.xml
<dataDir>${solr.data.dir:/usr/local/tomcat/solr/confoospeakers/data}</dataDir>

27. Interacting Solr using PHP
Apache Solr PECL Extension

28. How to Get the Solr PECL extension
pecl install solr-beta
Extract tarball
Enter extension directory
phpize
./configure
make and make install
adjust php.ini settings
run php -me

29. Adding Documents to Solr
$options = array (
'hostname' => SOLR_SERVER_HOSTNAME,
'port' => SOLR_SERVER_PORT,
'path' => SOLR_PATH_SPEAKERS,
'timeout' => SOLR_SERVER_TIMEOUT, );
/* Creating SolrClient instance */
$client = new SolrClient($options);

30. Adding Documents to Solr
/* Creating new input document */
$doc = new SolrInputDocument();
$doc->addField('speaker_id', $speaker_id);
$doc->addField('speaker_name', $speaker_name);
$doc->addField('company', $company);
/* Adding document to the index */
$client->addDocument($doc);
/* Finalizing Changes Do not forget this step */
$client->commit();

31. Removing Documents from Solr
/* Creating SolrClient instance */
$client = new SolrClient($options);
/* Remove the target document if you know the UniqueKey */
$client->deleteById($speaker_id);
/* Finalizing Changes. Do not forget this step */
$client->commit();

32. Searching for Data in the Index
SolrQuery Syntax
q=PHP
q=company:Microsoft
q=number_talks:[1 TO 3]
q="apache solr"~10

33. Searching for Data /* Creating SolrClient instance */
$client = new SolrClient($options);
$query = new SolrQuery();
$query->setQuery($search_string);
$query_response = $client->query($query);
$response = $query_response->getResponse();

34. Highlighting Hits /* Creating SolrClient instance */
$client = new SolrClient($options);
$query = new SolrQuery();
$query->setQuery($search_string);
$query->setHighlight(true);
$query->setHighlightUsePhraseHighlighter(true);
$query->setHighlightMaxAnalyzedChars(10000);
$query->setHighlightFragsize(5000);
$query->addHighlightField('speaker_name_t');
$query->addHighlightField('company_t');
$query->addHighlightField('bio');
$query->setHighlightSimplePre('<strong>');
$query->setHighlightSimplePost('</strong>');
$query_response = $client->query($query);
$response = $query_response->getResponse();

35. Dynamic Faceting of Results
/* Creating SolrClient instance */
$client = new SolrClient($options);
$query = new SolrQuery();
$query->setQuery($search_string);
$query->setFacet(true);
$query->setFacetMinCount(1);
$query->addFacetField('company');
$query->addFacetField('number_talks');
$query_response = $client->query($query);
$response = $query_response->getResponse();

36. AutoSuggest when Searching
/* Creating SolrClient instance */
$client = new SolrClient($options);
$query = new SolrQuery();
$auto_complete_type = 'terms';
if ('terms' == $auto_complete_type)‏ {
$query->setTerms(true);
if (strlen($auto_suggest_string))‏
$query->setTermsPrefix($auto_suggest_string); }
$query->setTermsField('speaker_name');
} else {
$query->addField('speaker_name');
$query->setQuery("{!prefix f=speaker_name}$auto_suggest_string");

37. Downloads and Feedback