{"id":690,"date":"2020-03-18T08:10:25","date_gmt":"2020-03-18T08:10:25","guid":{"rendered":"https:\/\/www.aeologic.com\/blog\/?p=690"},"modified":"2024-06-10T08:28:43","modified_gmt":"2024-06-10T08:28:43","slug":"all-about-indexing-and-basic-data-operations-part-2-ultimate-solr-guide","status":"publish","type":"post","link":"https:\/\/www.aeologic.com\/blog\/all-about-indexing-and-basic-data-operations-part-2-ultimate-solr-guide\/","title":{"rendered":"All About Indexing and Basic Data Operations – Part 2 – Ultimate Solr Guide"},"content":{"rendered":"

Hello Everyone! Today we are here with another post furthering our discussion about basic indexing operations in solr. The most commonly used form of data representation is JSON and XML. Today we will discuss how to handle indexing of custom JSON objects in solr. In order to do this, we use certain tags telling solr’s binary’s as to what has to be done and send them using update request. These parameters essentially handle the incoming JSON strings.One or more valid JSON documents can be sent to the\u00a0\/update\/json\/docs<\/code> path with the configuration params.<\/p>\n

Mapping Parameters<\/h2>\n

These parameters allow you to define how a JSON file should be read for multiple Solr documents.<\/p>\n

split<\/strong><\/p>\n

Defines the path at which to split the input JSON into multiple Solr documents and is required if you have multiple documents in a single JSON file. If the entire JSON makes a single Solr document, the path must be \u201c\/<\/code>\u201d.<\/p>\n

\n

It is possible to pass multiple\u00a0split<\/code>\u00a0paths by separating them with a pipe\u00a0(|)<\/code>, for example:\u00a0split=\/|\/hello|\/hello\/world<\/code>. If one path is a child of another, they automatically become a child document.<\/p>\n

f<\/strong><\/p>\n

Provides multivalued mapping to map document field names to Solr field names. The format of the parameter is target-field-name:json-path<\/code>, as in\u00a0f=first:\/first<\/code>. The\u00a0json-path<\/code>\u00a0is required. The\u00a0target-field-name<\/code>\u00a0is the Solr document field name, and is optional. If not specified, it is automatically derived from the input JSON. The default target field name is the fully qualified name of the field.<\/p>\n

mapUniqueKeyOnly<\/strong><\/p>\n

(boolean) This parameter is particularly convenient when the fields in the input JSON are not available in the schema and\u00a0schemaless mode<\/a>\u00a0is not enabled. This will index all the fields into the default search field (using the\u00a0df<\/code>\u00a0parameter, below) and only the\u00a0uniqueKey<\/code>\u00a0field is mapped to the corresponding field in the schema. If the input JSON does not have a value for the\u00a0uniqueKey<\/code>\u00a0field then a UUID is generated for the same.<\/p>\n

df<\/strong><\/p>\n

If the\u00a0mapUniqueKeyOnly<\/code>\u00a0flag is used, the update handler needs a field where the data should be indexed to. This is the same field that other handlers use as a default search field.<\/p>\n

srcField<\/strong><\/p>\n

This is the name of the field to which the JSON source will be stored into. This can only be used if\u00a0split=\/<\/code>\u00a0(i.e., you want your JSON input file to be indexed as a single Solr document). (Ultram<\/a>) Note that atomic updates will cause the field to be out-of-sync with the document.<\/p>\n

echo<\/strong><\/p>\n

This is for debugging purpose only. Set it to\u00a0true<\/code>\u00a0if you want the docs to be returned as a response. Nothing will be indexed.<\/p>\n

For example, if we have a JSON file that includes two documents, we could define an update request like this:<\/p>\n

\"\"<\/p>\n

With this request, we have defined that “exams” contains multiple documents. In addition, we have mapped several fields from the input document to Solr fields.<\/p>\n

When the update request is complete, the following two documents will be added to the index:<\/p>\n

\"\"<\/p>\n

In the prior example, all of the fields we wanted to use in Solr had the same names as they did in the input JSON. When that is the case, we can simplify the request by only specifying the\u00a0json-path<\/code>\u00a0portion of the\u00a0f<\/code>\u00a0parameter, as in this example:<\/p>\n

\"\"<\/p>\n

In this example, we simply named the field paths (such as\u00a0\/exams\/test<\/code>). Solr will automatically attempt to add the content of the field from the JSON input to the index in a field with the same name.<\/p>\n

ProTip<\/strong>: Documents will be rejected during indexing if the fields do not exist in the schema before indexing. So, if you are NOT using schemaless mode, you must pre-create all fields.<\/p>\n

Reusing Parameters in Multiple Requests<\/h3>\n

Say we wanted to define parameters to split documents at the\u00a0exams<\/code> field, and map several other fields. We could make an API request such as:<\/p>\n<\/div>\n

\"\"<\/p>\n

When we send the documents, we\u2019d use the\u00a0useParams<\/code>\u00a0parameter with the name of the parameter set we defined:<\/p>\n

\"\"<\/p>\n

Using Wildcards for Field Names<\/h2>\n
\n

Instead of specifying all the field names explicitly, it is possible to specify wildcards to map fields automatically.<\/p>\n<\/div>\n

\n

There are two restrictions: wildcards can only be used at the end of the\u00a0json-path<\/code>, and the split path cannot use wildcards.<\/p>\n<\/div>\n

\n

A single asterisk\u00a0*<\/code>\u00a0maps only to direct children, and a double asterisk\u00a0**<\/code>\u00a0maps recursively to all descendants. The following are example wildcard path mappings:<\/p>\n<\/div>\n

\n