Skip to main content

Posts

Showing posts from April, 2021

Databricks Integration with Azure Application Insight (Log4j) logs

In this cloud era, there is always the case where we need to capture our application logger logs to some central location. We are using Azure Application Insight to collect and analyze application logs. Reference – https://github.com/AnalyticJeremy https://medium.com/analytics-vidhya/configure-azure-data-bricks-to-send-events-to-application-insights-simplified-c6effbc3ed6a I would like to thank you Jeremy Peach for creating such a beautiful Listener and Balamurugan Balakreshnan for medium post. This post is the simplified summary of Spark Databricks application logs to Azure Application Insight. Step 1 – Get the required credentials, we need 3 things. Application Insights > Instrumentation Key Log Analytics workspace > Workspace ID Log Analytics workspace > Agents management > Linux servers > Primary key Step 2 – Clone the git repo git clone https://github.com/AnalyticJeremy/Azure-Databricks-Monitoring.git Step 3 – Edit appinsights_log