Confluent Cloud Series - Part 1 : Getting the .Net examples to work

Confluent Cloud is a great thing if you want to fiddle around with apache kafka without getting into the infrastructure too much.

In order to concentrate on the .Net stuff, I assume you'll figure out how to setup a cluster and topic yourself. Once that is done...

1. Log in to Cloud
2. Select Cluster

Once there, follow these directions:

1. Go to CLI & Client configuration
2. Go to Clients Tab
3. Create API Key and Secret and keep somewhere safe
4. Save the config to a file, say confluent.config. Replace Key and Secret. Bootstrap server should be set correctly, since this file was generated by the cloud (check, though).
5. Follow the example link (https://github.com/confluentinc/examples/tree/master/clients/cloud/csharp#overview)

Once there, you will see the documentation hinting that there are issues with certificates being accessible to the .Net library. So, as advised there, download Certificate from https://curl.haxx.se/ca/cacert.pem

Next, open Visual Studio and create a new .Net Core CMD solution and replace the program.cs with the one from the github link above. Add required packages (Confluent.Kafka, Newtonsoft.Json).
Finally add the config file and the certificate file to the project, edit the file properties of both to

• Build : "Content"
• Copy To output : "Always"

Now go to the settings of the project and add command line parameters as indicated by the docs.

• produce

This should have done the trick, but didn't. The issue here is that the configuration generated by the cloud is not formatted to fit the .net sample's needs.

So, either change the config file to match the pattern of the sample or adapt the code to match the string written in the actual configuration.

    static string ExtractJaasValue(string jaasConfig, string key)
    {
      var beginToken = key + @"=""";
      var startIdx = jaasConfig.IndexOf(beginToken);
      var endIdx = jaasConfig.IndexOf("\"", startIdx + beginToken.Length);
      return jaasConfig.Substring(startIdx + beginToken.Length, endIdx - startIdx - beginToken.Length);
    }

Now, you should be able to run this sample without further issues.

Learning resource Kafka and the role Zookeeper plays (paid but great): https://www.udemy.com/course/apache-kafka/