Confluent Kafka Security with OAuthBearer

Today we will try to explain one of the options of Confluent Kafka’s authentication mechanisms, the SASL OAuthBearer authentication with ACLs for authorization.

Confluent Kafka Security

Before jump to the OAuthBearer mechanism, it is important show the options that Confluent Kafka supports over security, below we classified the methods/components in three categories: Authentication, authorization and data encryption

Note that the data encryption works only in-transit from applications to brokers, the data sits unencrypted on the broker’s disk.

Encryption

You have to create SSL Keys and Certificates, configure brokers and applications, for details, check out the Confluent documentations.

It is important to observe that the Kafka relies on Zookeeper to store its metadata (maybe not for long), and the current version of Zookeeper do not support SSL/TLS, so it is important to protect the access and network of this important component.

OAuthBearer for Authentication

Below we have a diagram with the components and callbacks that must be implemented for the OAuth Bearer tokens retrieval, and simple implementation examples

source: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75968876

Callback Handlers

Server Callback Handler for Token Validation

Communication with OAuth2 Provider

Access Control List for Authorization

With the OAuthBearer authentication, the ACLs authorizer implementation uses the principal name exposed by org.apache.kafka.common.security.oauthbearer.OAuthBearerToken (below) to allow or deny access to the topics. In the callback handler we used the property sub (subject - JSON Web Token) from OAuth token introspection response as principal name, but could be customized as necessary.

Next Steps

To help this journey with a more in depth concepts, we recommend reading the Confluent Security Documentation.

Cloud Specialists providing professional services with DevOps, BigData, Cloud Native Applications and Security. https://www.3bit.com.br/3bit

Cloud Specialists providing professional services with DevOps, BigData, Cloud Native Applications and Security. https://www.3bit.com.br/3bit