Highlights of RisingWave v1.4: The Open-Source Streaming Database

The RisingWave team continues to deliver impressive features and updates with this new v1.4 release. This release continues to introduce new upstream and downstream connectors while improving upon existing ones. New functions and operators have also been added to allow for more complex data transformations. Follow along as we highlight some of the more notable features.

If you are interested in the full list of v1.4 updates, see the release note.

A new version of the S3 source connector

This release of RisingWave introduces a new version of the existing AWS S3 source connector. This improved version of the S3 source connector aims to resolve the scalability issues that the previous connector faced. The method of creating an S3 source with the improved connector is almost exactly the same as before. If you are interested in testing the integration between AWS S3 and RisingWave, we recommend using the upgraded version of the S3 source connector for a stable, hassle-free experience.

For more details:

See Ingest data from S3.
See the design document.

New sink connectors

The RisingWave team continues to improve RisingWave’s ability to connect to more systems, allowing RisingWave to be easily incorporated with any existing data stack. Check out our Integrations page for a list of supported systems. You can also vote to signal support for a specific integration to prioritize its development. With this release, RisingWave can sink data to Doris, Google BigQuery, and Redis. BigQuery, Doris, and Redis can all be used as a data storage tool. Using a simple SQL command, you can easily create a sink in RisingWave to output data to all these systems.

For more details:

See Doris sink documentation.
See Google BigQuery sink documentation.
See Redis sink documentation.
See Integrations page.

Support for COMMENT ON command

We now offer support for the COMMENT ON command. With this command, you can add comments to tables and columns, providing you with the ability to add documentation to the database objects. For current and future collaborators, the extra comments may allow them to quickly understand the purpose and usage of the tables and columns.

The following SQL query adds a comment to the table t1.

COMMENT ON TABLE t1 IS 'this is table 1'; 

To see the comment made, you can select from the rw_description RisingWave catalog.

SELECT * FROM rw_description; 
------- 
objoid | classoid | objsubid | description 
-------+----------+----------+------------------- 
  1001 |       41 |          | this is table 1 

You can also use the SHOW COLUMNS or DESCRIBE commands for the same purpose.

For more details:

See the documentation for the COMMENT ON command.

Support for subqueries in UPDATE and DELETE commands

You can now include a subquery when using the UPDATE or DELETE command. This allows for more complex data manipulations and easier data clean-up as you are able to perform updates and deletions based on specific conditions.

UPDATE table_name   SET col_1 = col_1 * 2.2 
  WHERE col_1 NOT IN (SELECT col_2 FROM drop_vals); 

For more details:

See the documentation for theUPDATE clause.
See the documentation for the DELETE clause.

A multitude of new JSON functions and operators

These new JSON functions and operators introduced in RisingWave handle JSONB types as inputs or outputs. In RisingWave, JSONB is a data type that enables you to create a column that stores JSON data. If being able to JSONB data types improves your data cleaning and transformation experience, these new features may come in handy.

This release introduces numerous JSON operators, allowing you to easily manipulate JSON data in RisingWave. Here are some examples of the new JSON operators:

The @> operator, which checks if the left JSONB object contains the values that are in the right JSONB object.

'[1, 2, 3]'::jsonb @> '[1, 3]'::jsonb → t

The - operator, which deletes a key-value pair, all matching keys, array elements, or an element at the specified index from a JSON object.

'{"a": "b", "c": "d"}'::jsonb - 'a' → {"c": "d"}

The #- operator, which deletes the element at the specified path. The path element can be field keys or array indexes.

'["a", {"b":1}]'::jsonb #- '{1,b}' → ["a", {}]

The full list of operators includes: @>, <@, ?, ?| , ?&, #>,  #>>, - and #-.

Four new JSON functions have also been added and they include jsonb_extract_path, jsonb_object, jsonb_pretty, and jsonb_strip_nulls. We will briefly introduce the jsonb_object and jsonb_pretty functions.

jsonb_object turns the input array of text elements and returns a JSONB object, where adjacent values in the string are read as key-value pairs.

jsonb_object(array['a', null]) → {"a": null}

jsonb_pretty outputs the input JSONB object as a nicely formatted text.

SELECT jsonb_pretty('[{"f1":1,"f2":null}, 2]');
------RESULT
[
    {
        "f1": 1,
        "f2": null
    },
    2
]

For more details:

See the documentation on JSON functions and operators.
See the JSONB data type.