Role of Big Data by Aliasgher Dalal, Director of Architecture – Data Analytics, Sigma Software Solutions

Big Data is garnering a lot of attention these days. The long-term value of big data with analytics is well understood and promises to capture a larger and larger share of business services. The billing systems appear to similarly benefit from the advent of big data. In fact, telecommunication systems, where large volumes of data are generated, are well suited for big data. Analytics play an important role in billing systems by providing insights into the operational, marketing and financial performance of the operator’s system, allowing executives to make data driven decisions.

Marketing needs ways to generate a wide array of analytical insights rapidly on the fly and cost effectively. Traditional approaches using Data marts or Data warehouses fall far short of meeting these expectations.

The traditional analytics approach poses several challenges. For one, the traditional approach is bound by rigid data models requiring extensive data definitions. A typical process to obtain useful analytics in a traditional data mart solution is as follows:

·        Define use case(s),

·        Identify associated data source(s)

·        Define data model

·        Develop ETL solution to ingest and populate

·        Develop queries to obtain analytics

In summary, the traditional analytics approach is inflexible, cost prohibitive (i.e. resource intensive), time consuming and out of synch with present day demands.In addition, it is not suited to manipulate large amounts of data, especially if the analytics must be obtained on streams of real-time data compared to archived data. 

On the other hand, big data offers capabilities that promise to handle these challenges effectively.

·         Ability to handle large  volume of data – data approaching tera, peta, exa and zeta bytes are handled using big data technologies

·         Ability to handle real-time analytics – big data technologies like Spark can perform useful analytics on data in real-time.

·         Flexible and dynamic – non-traditional database with dynamic schemas and flat structures allow data from varied sources to be ingested and made available for analytics without having a clear understanding of specific use cases. The analytics can be run on data as use cases are identified. This is a very flexible and inexpensive approach to generate analytics.

·         Cost effective – initial data preparation work is substantially reduced thereby making the big data approach cost effective.