A Day in the Life of Data Operations
April 20, 2017
I’ve been with Comlinkdata for almost three years now. I’m here in sort of a hybrid role, both a member of the data operations team as an analyst, and also as the manager of IT systems. Here, I’ll be focusing on the former. It was my first job out of college, a referral from a friend who already worked on the (much smaller) data team. I was looking for a place to put my computer and technology skills to use and I was happy to join a smaller company where I could have a larger impact. Before I joined, I hadn’t heard even heard of Comlinkdata, so let me get the ‘who we are’ out of the way. “Comlinkdata is the leading provider of telecom market data and insights. We provide clients with unique, real-time, query ready data that is combined with our analysts’ telecom expertise. At Comlinkdata, we help you make data-driven business decisions with confidence. Our data and insights provide you with the tools you need to analyze and optimize your business strategy ranging from decisions based on network investments, to pricing to market positioning.”
Matt Caplan, Analyst, Data Operations and IT ManagerClick here to view our open positions
Now, you can imagine with a company description like that, data analysis is important around here. “What does data operations do?” is a question you’ve probably always asked yourself, perhaps even been kept awake at night by. Finally, you’ll get the answers you’ve been seeking. A five-person team, we do the data cleaning, analysis, and integration that makes Comlinkdata so great. To do all this, we have regular daily or weekly responsibilities, ad hoc checks and fixes, and work on new projects.
The first thing I do (almost) every day is kick off an R script I wrote a couple of years ago. The script is simple – R executes SQL queries and combines the data into a clean Excel workbook, ready for analysis. The process is designed to automatically pull out abnormal data. Through a proprietary process developed by the team, we make changes as necessary and upload the clean data back to the servers, ready to be used to cleanse our datasets. A similar process is repeated with our wireless and Canadian data, with some variation in technique. Another team member will be reviewing new tickets and our testing procedures.
Our ad hoc work often involves working with other teams to confirm or check potential issues. Often, events such as mergers, network changes, promotions, or natural disasters can have very large effects on data patterns. We may be called in to verify that a rise or fall in activity is related to such an event. Once, we were asked to examine a sharp rise in one carrier’s market share in Louisiana. With some analysis and research, we determined the trend to be natural: major flooding was occurring, and while that carrier was offering free service to effected users, their largest competitor had a major outage. While ad hoc analysis can often be because of interesting events like this, it can also simply be abnormal data that wasn’t caught by one of our detection methods. In those cases, the data is cleansed and that process is used to improve detection methods in the future.
Our projects take many forms. Some of us have worked to help build out a validation process for our newer Canadian dataset. Other times, projects involve evaluating new data sources and incorporating them when they prove valuable. We work to rewrite and improve our detection methods and to test data changes. Reproducibility is an absolute must, which means documentation is a vital and ongoing need. Constant mergers, network expansions and other factors make regular research necessary too.
Data operations is an integral part of any company, a data analytics company especially. While we rarely interact with clients, we’re involved with most platforms, data sets, and services Comlinkdata offers. Hopefully this answered your burning question and you’ve come away with a deeper understand of the role data operations plays.