We all know that in today’s world, data is the king, but we have data in different forms, types, relational, non-relational, structured, unstructured manner. It becomes difficult to process the data and keep it in a single format that can be easily readable by the system, analytics and with proper context. ADF comes to the rescue here.

A pipeline in Azure Data Factory (ADF) is a logical grouping of activities that together perform a task. In this article, we are going to learn the steps to create a ADF pipeline.

In Azure Data Factory (ADF), obtaining the row count from a data source involves using a combination of activities within a pipeline. The most common method is to use a Lookup activity to execute a query that counts the rows in the dataset. Here's a step-by-step guide to do this:

In this article, we will see how you can stop the already running pipeline in Azure Data Factory (ADF).

Integration Runtime (IR) in Azure Data Factory is essentially the execution environment where the data movement, data transformation, and dispatch activities occur. Understanding the types of Integration Runtime and their specific use cases is a key to effectively leveraging Azure Data Factory.

Azure Data Factory (ADF) does not have a built-in activity to send emails directly (As of Feb 2024). However, you can achieve this functionality by integrating ADF with other Azure services that can send emails, such as Azure Logic Apps or Azure Functions. In this article, we will achieve that using Azure Logic Apps.

In this article, we are going to learn the steps of creating a Data Factory in Azure that can be done through the Azure Portal.

A Linked Service in Azure Data Factory (ADF) is a reference to a data store or a compute service. Think of it as a connection string that ADF uses to securely connect to external resources. Linked Services are essential for specifying how ADF should connect to various sources and destinations of data, as well as computing environments for data processing.

In this article, we are going to learn how we can trigger a logic app from ADF – Azure Data Factory.

The Switch activity in Azure Data Factory (ADF) allows conditional branching within your data integration workflows, like a "switch" or "case" statement in traditional programming languages. It enables you to direct the flow of execution to different paths based on the value of a specified expression. This is particularly useful for scenarios where you need to execute different logic based on the outcome of previous activities, the value of pipeline parameters, or other dynamic data.

The Lookup activity in Azure Data Factory is designed for retrieving a dataset from any supported data source. It can be particularly useful when you need to fetch configuration data or look up values that will be used in subsequent activities within your data pipeline.

The If Condition activity in Azure Data Factory (ADF) is a control flow activity that lets you specify two different sets of activities to be executed based on evaluating a Boolean expression. It is like an "if-else" statement in traditional programming, enabling conditional logic within your data integration workflows. This activity is particularly useful for dynamically controlling the flow of execution in your pipelines based on runtime conditions, data characteristics, or external parameters.

The Get Metadata activity in Azure Data Factory (ADF) is a powerful tool for obtaining metadata about data stored in various locations. This activity can be used to extract information such as file existence, file size, file count, and so on, before performing data integration tasks. This capability is crucial for conditional operations, like checking if a file exists before attempting to copy it, or for iterating over files in a directory.

The For Each activity in Azure Data Factory is designed to create a loop in your pipeline, allowing you to iterate over a collection and execute specified activities for each item in that collection. This is like the foreach loop structure in many programming languages. It is configurable for both sequential and parallel executions, supporting dynamic activity within loops by referencing the current item.

Azure Data Factory does not include a "Fail" activity as a built-in feature, like Copy or Filter activities. However, you can design failure scenarios manually using expressions and conditions to control the flow of your pipelines.

The Filter activity in Azure Data Factory is used to apply a filter expression to an input array, enabling conditional filtering of data within a pipeline. This activity allows for the selection of specific data that meets certain criteria, making it a versatile tool for data processing workflows where only a subset of data is required for subsequent activities.

The Until activity in Azure Data Factory (ADF) is a control flow activity that allows you to repeatedly execute a set of activities until a specified condition is met. It is like a "do-while" loop in traditional programming languages. The Until activity is particularly useful for scenarios where you need to perform operations such as polling an external system for a status update, retrying operations until they succeed, or processing data in chunks until all data has been processed.

The Web Activity in Azure Data Factory (ADF) is a versatile pipeline activity used for invoking HTTP endpoints. It can call REST APIs, trigger Azure Logic Apps or Function Apps, make HTTP requests to external services, and much more. This activity is useful for integrating external processes, data sources, or services into your data integration workflows.

The Execute Pipeline activity in Azure Data Factory allows one pipeline to invoke another pipeline within the same Data Factory environment.

In Azure Data Factory (ADF), a Delete Activity is used to remove files or folders from a file-based store (like Azure Blob Storage, Azure Data Lake Storage, Amazon S3, SFTP, etc.) based on specified conditions. This activity is useful for cleaning up data that is no longer needed as part of a data processing workflow.

In this article, we are going to learn basic things about logic apps, why we need logic app and all the basic areas of understanding about Logic Apps.

In this article we will see what do triggerBody() and triggerOuput() function do. How to use them, what is the difference between them and how to utilize their out put value in the logic app.

In this article, we are going to see a few very useful string manipulation functions that we can use in Logic Apps.

Like in any other application, in Logic apps also you would want one or more values that you can supply externally as a variable. In this article, we are going to see how we can utilize parameters in Logic apps.

There are some basic exceptions like request time out or edit conflicts between multiple simultaneous requests, that can be resolved by simple retry logic in the application. In this article, we will go in details of the retry policy in Azure Logic apps. A retry policy applies to intermittent failures, characterized as HTTP status codes 408, 429, and 5xx, in addition to any connectivity exceptions. The default is an exponential interval policy set to retry 4 times.

In this article, we are going to see how you can handle HTTP requests in logic apps and use the response from that request in the subsequent action of the logic app flow.

In this article, we will see how to parse and use JSON in Logic apps.

We must have faced a lot of issues while programming and what we used to do is apply try catch finally block. The azure logic app also comes with features using which we can apply error handling on logic app actions. We might face some data issues or system downtime issues because of which the running logic app can fail. Error and exception handling tools in the Azure logic app help us to build robust applications.