News - 16 series — Groonga v16.1.0 documentation

Skip to main content
`Ctrl`+`K`

# News - 16 series

## Release 16.1.0 - 2026-08-28

### Fixes

#### Fixed a crash bug when allocation of a new segment fails

This issue may occur when updating the index in an environment with insufficient memory or when the limit on the number of memory mappings has been reached.

Groonga outputs an error message when it cannot allocate memory while updating the index. However, the error message refers to an uninitialized value when it is generated.

This error message is generated only when Groonga cannot allocate memory due to insufficient memory or when the limit on the number of memory mappings has been reached. Therefore, this issue does not occur in an environment with sufficient memory.

## Release 16.0.9 - 2026-08-07

### Improvements

#### [benchmark] Remove incorrect `G_GNUC_CONST` usage

GH-2857 Reported by correctmost.

GH-2858 Patched by correctmost.

Changed for compatibility with recent GCC versions. See https://gitlab.gnome.org/GNOME/glib/-/commit/016829559 for details.

#### [string_truncate] Added new function `string_truncate()`

`string_truncate()` truncates a string to at most the specified number of characters. If the string is truncated, the tail of the kept characters is replaced with an omission mark so that the result, including the omission mark, is `length` characters long, in the same way as Ruby on Rails’ `String#truncate`.

Execution example:

    plugin_register functions/string

    table_create Memos TABLE_HASH_KEY ShortText
    load --table Memos
    [
    {"_key": "Groonga is a full text search engine"}
    ]

    select Memos \
      --output_columns '_key, string_truncate(_key, 15)'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         1
    #       ],
    #       [
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "string_truncate",
    #           null
    #         ]
    #       ],
    #       [
    #         "Groonga is a full text search engine",
    #         "Groonga is a..."
    #       ]
    #     ]
    #   ]
    # ]

#### Added new function `binary_length()`

`binary_length()` returns the size of the data in a column in bytes. You can use it to check the data size of each column as shown below.

    plugin_register functions/binary

    table_create Memos TABLE_HASH_KEY ShortText

    load --table Memos
    [
    {"_key": "ぐるんが"},
    {"_key": "Groonga"}
    ]

    select Memos \
      --output_columns '_key, binary_length(_key)'
    #[
    #  [
    #    0,
    #    1786079749.354771,
    #    0.0005497932434082031
    #  ],
    #  [
    #    [
    #      [
    #        2
    #      ],
    #      [
    #        [
    #          "_key",
    #          "ShortText"
    #        ],
    #        [
    #          "binary_length",
    #          null
    #        ]
    #      ],
    #      [
    #        "ぐるんが",
    #        12
    #      ],
    #      [
    #        "Groonga",
    #        7
    #      ]
    #    ]
    #  ]
    #]

#### [normalizers] Added support for Unicode 17.0.0

### Fixes

#### Fixed a bug where input was unintentionally tokenized while ignoring whitespace when using multiple normalizers

GH-2853 Reported by askdkc.

For example, the following text incorrectly matches the query “sayuri” cause of this issue:

    nanakusa yurine konnyaku

However, the text does not contain “sayuri”.

This issue occurs only when using multiple normalizers as shown below:

    table_create Normalizations TABLE_PAT_KEY ShortText
    column_create Normalizations normalized COLUMN_SCALAR ShortText
    load --table Normalizations
    [
    {"_key": "Anything", "normalized": "Whatever"}
    ]

    table_create EntriesChained TABLE_NO_KEY
    column_create EntriesChained content COLUMN_SCALAR Text

    table_create TermsChained TABLE_PAT_KEY ShortText \
      --default_tokenizer 'TokenNgram("unify_alphabet", false)' \
      --normalizers 'NormalizerNFKC150, NormalizerTable("normalized", "Normalizations.normalized")'
    column_create TermsChained content_index COLUMN_INDEX|WITH_POSITION EntriesChained content

    load --table EntriesChained
    [
    {"content": "nanakusa yurine konnyaku"}
    ]

    select EntriesChained \
      --match_columns content \
      --query sayuri \
      --output_columns _id,content
    [
      [
        0,
        1783649374.310488,
        0.0007901191711425781
      ],
      [
        [
          [
            1
          ],
          [
            [
              "_id",
              "UInt32"
            ],
            [
              "content",
              "Text"
            ]
          ],
          [
            1,
            "nanakusa yurine konnyaku"
          ]
        ]
      ]
    ]

### Thanks

- askdkc

- correctmost

## Release 16.0.8 - 2026-07-13

### Improvements

#### [between] Added support for vector type value as the 1st parameter

> Note that `between` does not use an index for a vector value yet.

In the following example, `between` returns `true` if any of the elements is included in the specified range. Therefore, we can use `between` to search records that have at least one value in the range.

Execution example:

    table_create Products TABLE_HASH_KEY ShortText
    # [[0,1337566253.89858,0.000355720520019531],true]
    column_create Products prices COLUMN_VECTOR Int32
    # [[0,1337566253.89858,0.000355720520019531],true]

    load --table Products
    [
    {"_key": "A", "prices": [17, 170, 1700]},
    {"_key": "B", "prices": [18, 180, 1800]},
    {"_key": "C", "prices": [19, 190]},
    {"_key": "D", "prices": [20]},
    {"_key": "E", "prices": [21, 210, 2100]}
    ]
    # [[0,1337566253.89858,0.000355720520019531],5]

    select Products --filter 'between(prices, 18, "include", 20, "exclude")'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         2
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "_key",
    #           "ShortText"
    #         ],
    #         [
    #           "prices",
    #           "Int32"
    #         ]
    #       ],
    #       [
    #         2,
    #         "B",
    #         [
    #           18,
    #           180,
    #           1800
    #         ]
    #       ],
    #       [
    #         3,
    #         "C",
    #         [
    #           19,
    #           190
    #         ]
    #       ]
    #     ]
    #   ]
    # ]

#### Dropped support for Debian 12 (bookworm)

It reached EOL on 2026-06-10.

### Fixes

#### Fixed a crash bug when we specify `language_model_knn(...)` as `sort_keys`

For example, if you specify `language_model_knn(text, "male child", {"k" : 2 })` as `sort_keys`, Groonga crashes due to a double free that occurs while processing the `{ "k" : 2 }` option in the sort routine.

#### Fixed a crash bug when we use `language_model_knn()` with empty value

Groonga crashes when `language_model_knn()` is used with empty value as shown below:

    plugin_register language_model/knn
    [[0,0.0,0.0],true]

    table_create Data TABLE_NO_KEY
    [[0,0.0,0.0],true]
    column_create Data text COLUMN_SCALAR ShortText
    [[0,0.0,0.0],true]
    column_create Data rabitq_code COLUMN_SCALAR ShortBinary
    [[0,0.0,0.0],true]

    load --table Data
    [
    {"text": "I am a boy."},
    {"text": ""},
    {"text": "This is an apple."}
    ]
    [[0,0.0,0.0],3]

    table_create RaBitQ TABLE_HASH_KEY ShortBinary \
      --default_tokenizer 'TokenLanguageModelKNN("model", \
                                                 "hf:///groonga/all-MiniLM-L6-v2-Q4_K_M-GGUF", \
                                                 "code_column", "rabitq_code")'
    [[0,0.0,0.0],true]
    column_create RaBitQ data_text COLUMN_INDEX Data text
    [[0,0.0,0.0],true]

    select Data   --filter 'language_model_knn(text, "male child")'   --output_columns text

### Experimental features

> This is an experimental feature. Currently, this feature is still not stable.

#### [json_extract] Added support for `json_extract()`

`json_extract()` extracts values from a JSON by a JSONPath expression. The extracted values keep their JSON types. For example, a string in JSON is extracted as a text and an integer in JSON is extracted as an integer. Therefore, we can use `json_extract()` for both full text search against strings and range search against numbers as shown below:

    table_create Data TABLE_NO_KEY
    # [[0,1337566253.89858,0.000355720520019531],true]
    column_create Data value COLUMN_SCALAR JSON
    # [[0,1337566253.89858,0.000355720520019531],true]

    load --table Data
    [
    {"value": "{\"value\": [[1, 10], [100]]}"},
    {"value": "{\"value\": [[2], [20, 200]]}"},
    {"value": "{\"value\": [[-1, -10], [-100]]}"}
    ]

    select Data --filter 'between(json_extract(value, "$.value[*][*]"), 10, 20)'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         2
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "value",
    #           "JSON"
    #         ]
    #       ],
    #       [
    #         1,
    #         {
    #           "value": [
    #             [
    #               1,
    #               10
    #             ],
    #             [
    #               100
    #             ]
    #           ]
    #         }
    #       ],
    #       [
    #         2,
    #         {
    #           "value": [
    #             [
    #               2
    #             ],
    #             [
    #               20,
    #               200
    #             ]
    #           ]
    #         }
    #       ]
    #     ]
    #   ]
    # ]

#### [ExtractorJSON] Added support for `ExtractorJSON`

> This is an experimental feature. Currently, this feature is still not stable.

`ExtractorJSON` extracts values from JSON data by a JSONPath expression as well as `json_extract()`. We can use this extractor to index only the values you need from JSON without indexing the whole JSON text.

When you attach `ExtractorJSON` to a lexicon, the lexicon indexes the extracted values. The original JSON is kept in the data column.

The following example indexes integers in a JSON column. The lexicon key type is Int32 because the extracted values are integers. The index is used automatically when the JSON column is loaded, so you can search the original records by the extracted values:

    table_create Data TABLE_NO_KEY
    # [[0,1337566253.89858,0.000355720520019531],true]
    column_create Data value COLUMN_SCALAR JSON
    # [[0,1337566253.89858,0.000355720520019531],true]
    table_create Numbers TABLE_PAT_KEY Int32 \
      --extractors 'ExtractorJSON("path", "$.value[*][*]")'
    # [[0,1337566253.89858,0.000355720520019531],true]
    column_create Numbers data_value COLUMN_INDEX Data value
    # [[0,1337566253.89858,0.000355720520019531],true]
    load --table Data
    [
    {"value": "{\"value\": [[1, 10], [100]]}"},
    {"value": "{\"value\": [[2], [20, 200]]}"},
    {"value": "{\"value\": [[-1, -10], [-100]]}"}
    ]
    # [[0,1337566253.89858,0.000355720520019531],3]
    select Data --filter 'between(Numbers.data_value, 10, 20)'
    # [
    #   [
    #     0,
    #     1337566253.89858,
    #     0.000355720520019531
    #   ],
    #   [
    #     [
    #       [
    #         2
    #       ],
    #       [
    #         [
    #           "_id",
    #           "UInt32"
    #         ],
    #         [
    #           "value",
    #           "JSON"
    #         ]
    #       ],
    #       [
    #         1,
    #         {
    #           "value": [
    #             [
    #               1,
    #               10
    #             ],
    #             [
    #               100
    #             ]
    #           ]
    #         }
    #       ],
    #       [
    #         2,
    #         {
    #           "value": [
    #             [
    #               2
    #             ],
    #             [
    #               20,
    #               200
    #             ]
    #           ]
    #         }
    #       ]
    #     ]
    #   ]
    # ]

## Release 16.0.5 - 2026-05-22

### Fixes

#### Fixed a bug where Groonga for Windows failed to run due to a missing required DLL

The required DLL (`msvcp140_atomic_wait.dll`) for Groonga for Windows was missing from the `groonga-16.0.2-x64-vs2022-with-vcruntime.zip`.

This issue only affects Groonga 16.0.2 for Windows.

## Release 16.0.2 - 2026-05-07

### Improvements

#### Added support for Ubuntu 26.04 LTS

### Fixes

#### Fixed the ODR(One Definition Rule) violation

GH-2787 Reported by Nicolas PARLANT.

The LTO(Link-Time Optimization) build was failing because `grn_tokenizer_query` had two different definitions. This change fixes the ODR violation, allowing the LTO process to complete successfully.

#### Fixed missing files in benchmark directory of source archive

GH-2793 Reported by Nicolas PARLANT.

The missing files are as follows:

- groonga/benchmark/CMakeLists.txt

- groonga/benchmark/bench-distance.c

- groonga/benchmark/geo-distance-summary.rb

#### Fixed a bug where duplicate “tools” directories were created

GH-2798 Reported by Kentaro Hayashi.

Fixed an issue where duplicate “tools” directories were created as below when installing the groonga-tools package on AlmaLinux and Amazon Linux:

    /usr/share/groonga/tools/tools/

### Thanks

- Nicolas PARLANT

- Kentaro Hayashi

## Release 16.0.1 - 2026-03-30

### Improvements

#### [language_model_vectorize] Added `prefix` option

You can now add a prefix to the input text. This is useful for models that require a prefix, similar to the `passage_prefix` and `query_prefix` options of `TokenLanguageModelKNN`.

    language_model_vectorize("hf:///groonga/multilingual-e5-base-Q4_K_M-GGUF", \
                             "male child", \
                             {"prefix": "query: "})

#### [object_list] Added normalizer information to the output

`object_list` command now includes a `normalizers` field in the output for each table.

### Fixes

#### [HTTP] Fixed a false error for chunked HTTP requests

Fixed a bug where valid chunked HTTP requests could fail with an error under certain conditions.

This issue could only occur when a chunked request body was split across multiple receives in a very specific pattern, so most users would never encounter it. If it did occur during a `load` command, the load would only be partially completed and would need to be re-executed.

No index corruption occurs due to this bug.

#### Fixed a bug that caused a crash due to using the wrong free function

Fixed a bug where `grn_obj_unlink()` was used instead of `GRN_OBJ_FIN()` for bulk objects. This could cause a crash in some cases.

Reported by Daniel Black

See also: https://jira.mariadb.org/browse/MDEV-39098

### Experimental features

> These features are still experimental and unstable. We must not use these features in production.

#### [Data types] Added support for arrays and objects in `JSON` type

The `JSON` type now supports arrays and objects, including nested ones. All JSON value types are now supported.

#### Added support for OpenZL compression and decompression for scalar columns and Float32 vector columns

OpenZL compression and decompression are now supported for scalar columns and Float32 vector columns. OpenZL can achieve better compression ratios than Zstandard.

Examples of compression:

The compression of float32 array results for the following data:

- The number of array elements is 40,960.

- The number of records is 10,000.

Results:

- Compress with OpenZL: 1.3GB

- Compress with Zstandard: 1.5GB

- No compress: 1.6GB

#### Added extractor support

Extractors are a new type of module in Groonga that extract plain text from structured data before tokenization.

The built-in `ExtractorHTML` extractor strips HTML tags and decodes HTML entities from a value, leaving only the text content.

Here is an example of using `ExtractorHTML` with the `extract` command:

    extract \
      --extractors 'ExtractorHTML' \
      --value "<html><body>He&lt;ll&gt;o</body></html>"
    [[0,0.0,0.0],{"extracted":"He<ll>o"}]

### Thanks

- Daniel Black

## Release 16.0.0 - 2026-02-09

This is our annual major release! This release doesn’t have any backward incompatible changes! So you can upgrade Groonga without migrating your existing databases. You can still use your existing databases as-is.

### Fixes

#### Fixed a bug that causes overflow in the key of `TABLE_DAT_KEY` tables

If you set a value of 4096 byte against a key, a `TABLE_DAT_KEY` table may be broken.