Sdairs

Tag: cloudera

  • The Cloudera Innovation Accelerator & me

    I recently took on a new role at Cloudera as a Senior Developer Advocate in the newly formed Cloudera Innocation Accelerator. This exciting new team sits between Cloudera and the wider data community, looking at new & ermeging technologies that are in-demand & have real potential to provide value to Cloudera’s customers. In this role, […]

  • Getting started with NiFi’s ScriptedReader by processing Key Value Pairs – Part 1

    Records have become an integral part of working with NiFi since their introduction on May 8th, 2017 with the release of NiFi 1.2.0. They allow us to treat our data as more than just a bunch of bytes, giving NiFi the ability to better understand and manipulate common data formats used by other tools. However, […]

  • S3 Event Notifications in NiFi

    The desire to pull newly uploaded files from storage is common, and typically the ListFile -> FetchFile pattern has been used in NiFi to monitor for, and then pull, new files as they arrive. Taking NiFi in to the cloud means that we’re often working with cloud object storage, like Amazon S3. While we can […]

  • Importing individual flow definitions in NiFi

    While I recommend you Version Control & Deploy flows with NiFi Registry you can also quickly import flow definition json files straight from the NiFi GUI. This post will quickly demonstrate how. To get some json files to work with, clone my nifi-snippets repo that contains all of the NiFi flow demos that I have […]

  • Building a simple REST API in NiFi

    In a previous post I discussed using REST APIs to enrich records at the time of ingest. This post will cover building the corresponding REST API that I used in that post. This will be a very simple REST API that exposes a single endpoint GET /api. This endpoint returns a 200 OK response with […]

  • Kafka with multiple Listeners and SASL

    This will quickly discuss how to configure multiple Listeners, with the intent of having a unique Listener for External/Client traffic and another for Internal/Inter-broker traffic (and how this can be done with Cloudera Manager which requires a slight work-around in the current versions pre-2021). There’s several valid use cases for multiple Listeners. In this case, […]

  • Connecting to a secure Impala via NiFi with TLS and Kerberos (and Hive2)

    There’s a few different ways this could be done. I’ll demonstrate one possible way, using ExecuteSQL to connect to Impala via the JDBC driver.