There’s just an endless plethora of tools out there for the novice and the expert. Sometimes, it even seems like the open source community can’t catch a break until something new hits the web. This time, we’ll walk readers through another exciting addition to the Cloud Native Computing Foundation (CNCF) – Strimzi.
What Exactly is Strimzi
Originally inducted into the CNCF as a Sandbox Project back in 2019, Strimzi was created to help operators to manage the various components of another well known platform- Apache Kafka on Kubernetes. Readers may want to take a look back at our Kafka article to catch a better glimpse.
Strimzi became a standard for applications including Kafka brokers, Apache ZooKeeper, MirrorMaker and Kafka Connect. The end result was a better way to manage and control the Kafka platform and its native APIs, an issue that was commonly acknowledged by the community. One of these additions being the better management for topics and messages through HTTP, protocol supports, storage improvements and monitoring changes.
How Strimzi Ties In With Kafka
Kafka, a creation of Linkedin dating back to early 2010s, was a message streaming tool for handling queuing systems and helping users manage large datasets on-time for intersecting with multiple social channels. Kafka runs on a similar structure for dataset handling, akin to SQL type of streams, rebranded as its own name- KSQL. A common problem often emerging out of Kafka’s usage has been its limitations for providing access control and distributing resources properly.
Most Kubernetes applications use a stream of operators to package, deploy and manage different elements in Kubernetes. Strimzi closes the gap on many of these resource needs by having the operators extend functionalities and automating tasks related to Kafka deployment. Users can thus apply Kafka changes and code applications through simplified CLI or GUI, without the complications of manually changing settings.
Strimzi Functionalities: What Makes It So Unique
Strimzi functions as a repository for container images, closely composed of Kubernetes elements. It also contains operators as previously discussed for getting Kafka to the Kubernetes platform. All operators designed for Kafka through Strimzi serve a specific purpose and are designed to resolve issues with control and deployment. Strimzi Operators are fundamental to the running of Strimzi.
Here is a small list of what the operators on Strimzi accomplish:-
- Deploying and running Kafka clusters along with Kafka components through CLI commands as well as GUI interactions. Users can also now seamlessly connect Kafka clusters to other channels and perform changes on time, as opposed, to running with the default binary data formats that Kafka was notoriously designed to hold.
- Configuring access to Kafka, its subsidiary channels and also all its individual components. This is most useful for controlling KSQL databases and performing changes as required by the operation or project.
- Securing access to Kafka and helping users define protocols for specifying access control to repositories. A much needed introduction to the platform in the wake of multiple data breaches and vulnerabilities, Strimzi has a specialized platform for setting up keys and passwords for stream levels and assigns a hierarchy of safety checks based on user requirements.
- Given the lack of large experts and documentations for running Kafka, which may have led most novices to discount it compared to its alternatives, upgrading Kafka hs become a hassle free task and gives users a simple list of commands to get latest changes on the server. What’s even better is that updates can now go undisturbed in the background without interfering with the Kafka clusters, a common problem reported by several users in the past.
- Managing brokers and nodes that reside below and above the main stream edges. Having proper communication between the various elements of a Kafka network is important and Strimzi accomplishes that with a mix of interactive GUI and simple to use elements.
- Strimzi also helps users to create topics and manage user interactions on social streams which required a much higher level of coding with previous configurations. Sometimes users would even have to log into different accounts to manage multiple topics, which is now all neatly wrapped in a single tabular framework.
A Treasure Trove of Operators: How To Best Use Strimzi
Knowing how to best use Strimzi requires knowing best how each of the operators can be applied for different needs and purposes:-
The cluster operator is closely related to operations for deploying and managing Apache Kafka clusters, Kafka Connect, Kafka MirrorMaker, Kafka Bridge, Kafka Exporter, and the Entity Operator.
The cluster operator is also, by its name, the central operator to help connect and control higher and lower level nodes in the Kafka network. This applies to the KSQL databases as well for configuring data join commands and deployment features to streams.
Linked directly to the cluster operator and composed of the Topic Operator (manages Kafka Topics) and User Operator (manages Kafka Users), the entity operator can be regarded as an index level information for the databases, helping users know which streams are related to other streams.
It is also an important element for connecting multiple streams and broadcasting changes on time.
The central operator for deploying and managing Kafka topics, which in the context of social channels and streams, refers to elements or information indexes that are linked to larger streams.
Similar to the Topic Operator, this operator is linked to the former and helps users identify indexes for users linked to the topics that are related to the social streams. Users can easily update user requirements, change stream update frequencies, edit channel settings and so on.
It should be noted that the Cluster Operator can deploy the Topic Operator and User Operator as part of an Entity Operator configuration. What’s amazing is that all this can be achieved at the same time as a Kafka cluster.
A Much Tinier Issue: Where Some Work Is Still Needed
The fly in the ointment in the vast improvements that Strimzi brings to the table is the fact that it is still a native application platform that is very much open source. With a lack of commercial support and documentation for dealing with errors/issues, Strimzi requires a level of familiarity with the platform to master it perfectly.
While the community has made amazing strides in providing newcomers with the right tools and methods to best utilize Kafka and its streams, much is still left to be desired in terms of official releases, updates and compatibility issues.
Some criticism has also been lodged at the platform’s absolute dependence towards Kafka and its specific dataset structures which makes transfer to other formats difficult. Strimzi remedies this in some ways by using functionalities through the cluster operator but is still in extreme need of updates.
Final Notes and Review
All in all, Strimzi and Kafka seem to go well hand in hand, offering improvements for the much discussed platform. Strimzi as a cloud native application manages to hit the right notes in bringing the best features of Kafka, which were often left unexplored or unused altogether. Being a much younger name to the CNCF name however, does mean it has to go through the same process as its counterparts in the open source space to win over user engagement. Only time perhaps will tell if the community can bring its support and documentation to a more user friendly level.
Doing so would cut down the steep learning curve that Kafka already has. For more information on these two platforms, check out the official documentations and our articles, where we do a deeper analysis on similar tools or platforms. Feel free to contact us if you need help integrating Kafka with Strimzi.