Join us in exploring some of the highly anticipated prominent features in this new release, including advanced support for regular expression functions, a syntax update for creating Kafka, Kinesis, and Pulsar sinks, and more.
This new release of RisingWave (v1.3) is packed with new and valuable features for sinks, sources, and SQL functions. Notable enhancements include advanced support for regular expressions, capabilities for delivering data to more downstream systems, and the addition of the WITH ORDINALITY clause.
For the full list of updates, see the release note.
With this release, multiple new array functions were added. These new functions include array_min, array_max, array_sort, array_sum. Array functions accept array types as inputs. These differ from aggregate functions, which perform calculations on a set of values. For more details on the new functions, including examples, look for the functions listed above on the Array functions page.
We have introduced enhanced support for advanced features in our existing regular expression functions. These functions, including regexp_count, regexp_match, regexp_matches, and regexp_replace, now offer additional capabilities such as backreference, positive lookahead, negative lookahead, positive lookbehind, and negative lookbehind.
WITH ORDINALITY clause
The WITH ORDINALITY clause is now also supported. When using set returning functions, this clause can be used at the end of the FROM clause in a SELECT statement. It will add a new column to the query results, which numbers the rows returned by the function, starting from 1. For more details, see WITH ORDINALITY.
SELECT * FROM
unnest(array[0,1,2])
WITH ORDINALITY;
Previously, when creating a sink, the type of sink is specified under the WITH options with the type parameter. As an example, the following SQL query creates an append-only Kafka sink.
CREATE SINK s1 FROM mv WITH (
connector = 'kafka',
properties.bootstrap.server = 'localhost:9092',
topic = 'test_topic',
type = 'append-only');Now the type of sink and data format can be specified with FORMAT...ENCODE... to be consistent with CREATE SOURCE. Note that the append-only type has been updated to PLAIN. For now, only the JSON format is supported. The following SQL query creates an append-only Kafka sink. Note that PLAIN is specified instead of append-only.
CREATE SINK snk FROM mv with (
connector = 'kafka',
properties.bootstrap.server = 'localhost:9092',
topic = 'test_topic')
FORMAT PLAIN ENCODE JSON;This syntax change only applied to Kafka, Kinesis, and Pulsar sinks. For more details, see the Sink data to Kafka page or the Sink data to AWS Kinesis page.
This release introduces support for multiple new sink connectors, allowing you to sink data transformed in RisingWave into more systems. RisingWave strives to provide users the ability to deliver data to popular downstream systems. Check out our Integrations page for a list of supported systems. You can also vote to signal support for a specific integration to prioritize its development.
Cassandra and ScyllaDB are open-source NoSQL database management systems capable of handling large workloads. Since ScyllaDB is built as a replacement for Cassandra, the RisingWave sink connector for both are the same. See the Cassandra sink guide for more details.
Apache Doris is an open-source distributed analytical database designed for ad-hoc analysis of large volumes of data. See how simple the process is to deliver data from RisingWave to Doris with this guide.
Elasticsearch is an open-source distributed search and analytics engine designed to process and handle large values of data in real time. Follow our guide to see how to create an Elasticsearch sink in RisingWave.
NATS is an open-source messaging system, which follows a pub-sub and request-response messaging pattern, ideal for real-time applications. See the documentation for details on how to sink data to NATS.
Apache Pulsar is an open-source distributed messaging system, designed for developing real-time, event-driven applications. With RisingWave, you can now ingest data from and sink data to Pulsar. See our new guide on how to create a sink to deliver data to Pulsar.
Conclusion
These are just some of the new features included with the release of RisingWave v1.3. To get the entire list of updates, which includes new regular expression functions, please refer to the detailed release notes.
Look out for next month’s edition to see what new, exciting features will be added. Check out the RisingWave GitHub repository to stay up to date on the newest features and planned releases.
Sign up for our monthly newsletter if you’d like to keep up to date on all the happenings with RisingWave. Follow us on Twitter and Linkedin, and join our Slack community to talk to our engineers and hundreds of streaming enthusiasts worldwide.
In this article, we'll show you how to set up a continuous data pipeline that seamlessly captures changes from your Postgres database using Change Data Capture (CDC) and streams them to Apache Iceberg.
By combining platforms like EMQX for industrial data streaming and RisingWave for real-time analytics, manufacturers can tap into machine-generated data as it happens, enabling predictive maintenance, reduced downtime, and improved efficiency. This integrated approach allows industries to respond swiftly to equipment failures, optimize production, and make data-driven decisions that boost overall equipment effectiveness (OEE) and operational agility.
In this article, we’ve demonstrated how to build a core fraud detection system using RisingWave. With minimal setup, you can easily integrate these components into your existing technical stack and have a functional fraud detection solution up and running.
