WILMINGTON, DE — The Apache Software Foundation has elevated Apache Iggy and Apache Sourcelume to Top-Level Project status, giving the two open-source projects independent governance as developers confront growing infrastructure demands from real-time data and artificial intelligence systems.
Apache Iggy is a persistent message-streaming platform written in Rust and designed to process high-volume, real-time data with low latency and a relatively small infrastructure footprint. The platform supports TCP, QUIC, HTTP and WebSockets and provides software development kits for multiple programming languages.
The project uses a thread-per-core, shared-nothing architecture and Linux io_uring, an approach intended to reduce thread contention and other sources of processing delays. Iggy also includes a connector ecosystem and clustering based on Viewstamped Replication Revisited.
Kranti Parisa, chair of the Apache Iggy Project Management Committee, said the platform can process millions of messages per second and hundreds of terabytes of data per day on a single node while maintaining single-digit millisecond tail latency.
“From the beginning, we’ve had an obsession with performance, predictable tail latency, and infrastructure efficiency,” Parisa said, describing those attributes as core design principles for the project.
Top-Level Project status gives Iggy an independent, community-governed structure within the Apache Software Foundation as real-time data processing becomes increasingly important for data-intensive and AI workloads.
Apache Sourcelume addresses a different infrastructure issue emerging from the growth of AI: documenting the origins and permitted uses of training data.
The open-source project provides instrumentation for AI training-data provenance, allowing dataset producers, model developers and downstream users to document where training data originated and the terms governing its use.
Rather than establishing a separate dataset-description vocabulary, Sourcelume adds an attestation and registry layer to existing standards. It allows metadata to be cryptographically signed, published and independently checked.
“Training data is the foundation everything else in AI is built on, and right now that foundation is largely undocumented,” Jamie Goodyear, chair of the Apache Sourcelume Project Management Committee, said.
Goodyear described the project’s signed records as an accountability mechanism rather than an endorsement of the underlying data.
“A producer can state — in a standard, cryptographically signed way — exactly what they’re asserting, and anyone downstream can verify who said it and that it hasn’t changed,” Goodyear said.
Top-Level Project status within Apache signifies that a project operates under its own Project Management Committee and has progressed beyond the foundation’s incubation process. For Iggy and Sourcelume, the change establishes independent governance for technologies aimed at two expanding areas of computing infrastructure: real-time data movement and verifiable AI training-data provenance.
Support the local news that supports Chester County. MyChesCo delivers reliable, fact-based reporting and essential community resources—free for everyone. If you value that, click here to become a patron today.