IDS Conventions - elasticsearch.json

TetraScience has created a elasticsearch.json generating script. You can find our public repo here:

Platform Requirements

#RuleChecked by IDS Validator
1datacubes, if defined in schema.json, should be in nonSearchableFieldsYes

Design Guideline

#RuleChecked by IDS Validator (only for IDS designed by TS)
1All object-type fields in schema.json should be defined as nestedYes, warning if not followed


Fields defined here will not be indexed into Elasticsearch. You want to exclude fields when:

  • You will not likely to search this field
  • Save some space
  • ES indexing will be faster
  • Your file is larger than 100MB. ES indexing currently has a hard limit on 100MB.

nonSearchableFields uses lodash omit to exclude, so follow the official lodash syntax here:

Elasticsearch Concept - Nested

Elasticsearch official documentation on nested:

All object-type fields in schema.json should be defined as nested.

There are situations where you don’t want to define a field to be nested:


Nested - Example

If you have the following expected.json:

  "results": [{
    "peaks": [{
      "number": 1,
      "name": "a" 
      "number": 2,
      "name": "b"

Then, your elasticsearch.json should look like the following:

  "mapping": {
    "properties": {
      "result": {
        "type": "nested",
        "properties": {
          "peaks": {
            "type": "nested"
    "dynamic_templates": []
  "nonSearchableFields": []

Did this page help you?