A pipeline-based analytics system can be characterized by a flow of data that improves the speed and consistency of the relevant judgments that can be gained through analyzing information. It shares similarities with continuous integration/continuous delivery pipelines in that speed is accumulated due to the automation of certain tasks. Essentially, a pipeline creates a workflow of constant automatic updates to reports, databases, or other forms of systems that store, investigate, and analyze data (Morgan, 2021). The strengths of pipeline-based analytics are in the continuous flow of information updates, speed, and less required effort.
However, this also presents several challenges that are inherent to this form of analytics. For instance, data may need categorization that a single pipeline cannot produce and inadequate governance can lead to unusable insights or even faults that corrupt the available data. As such, two vital improvements are necessary to target the weaknesses that pipeline-based analytics currently present. First, a company requires a variety of pipelines that are responsible for varied data and purposes. This diversification will result in easier categorization and inherently provide clear insights. While this may supply more need for management and observation over the pipelines, it presents an advantage to data analytics that a smaller pipeline structure could not exemplify. Second, pipeline-based analytics must follow appropriate governance as mismanagement in the process may result in unusable data and systems that cannot be maintained efficiently. Current efforts that connect data engineers with additional departments have resulted in greater overall efficiency but have also reduced the availability of these specialized employees in maintaining adequate operations of data pipelines. A balance must be mediated before the installation and use of complex pipeline-based analytics.
Reference
Morgan, L. (2021). What makes up an analytics pipeline? TechTarget. Web.