Mid to Senior Level Data Engineer
Reports to: Director of Engineering
Location: Boston, MA
Comlinkdata is looking for an experienced Data Engineer to join our Research and Development team. If you have strong programming/problem solving skills, a desire to continue learning and a passion for developing and improving data analysis applications, then we want to speak with you. You’ll be challenged with improving our processes for working with large datasets, writing code for data processing and optimizing data analysis processes. You’ll also collaborate across Comlinkdata teams to ensure our data and products exceed client expectations, and to streamline and data operations.
Including but not limited to:
- Work with large datasets of hundreds of terabytes to process it within hours leveraging Spark, Hive and AWS Elastic MapReduce (EMR)
- Create new environments for product development teams to run statistical and ML models, data analysis and ETL processes
- Directly implement complex models and data pipelines on large datasets
- Work with the wider team to educate them on the value of good software development practices such as architecture design, clean code and clear naming conventions
- Implement data tests and QA processes
- Participate in cross-functional projects e.g. EMR functionality enhancements, analysis tool evaluation,
- 5-10 years of work experience in large data processing and analytics projects, bringing algorithms and proof-of-concepts to production
- Bachelors Degree in Computer Science, Mathematics, Engineering or similar
- Well trained in good programming practices
- Sound computer science fundamentals
- Extensive applied experience using Spark to develop and implement ETL processes and data pipelines
- Advanced knowledge of Python (or similar scripting language)
- Advanced knowledge of SQL (MSSQL, MySQL and T-SQL)
- Familiarity with data analysis and statistical modeling using Python (e.g. NumPy/SciPy, pandas) or R (or similar statistical language)
- Experience with Object-Oriented programming
- Self-motivated; capable of working independently and as part of a team
- Experience with AWS Environment or Similar Cloud Services
- Experience with Linux
- Experience with Java, C++ or C
- Database Design Experience
- Hadoop/EMR cluster tuning
- Test Driven Development
- ML algorithms and good ML implementation practices
- Application of ML techniques and other complex algorithms at large scale using Spark or similar
At this time, Comlinkdata will not sponsor a new applicant for employment sponsorship for this position.
Candidates must submit a resume and cover letter to be considered.
Comlinkdata is the leading provider of telecom market data and insights. We provide clients with unique, real-time, query ready data that is combined with our analysts’ telecom expertise. At Comlinkdata, we help you make data-driven business decisions with confidence. Our data and insights provide you with the tools you need to analyze and optimize your business strategy ranging from decisions based on network investments, to pricing to market positioning.
Comlinkdata is headquartered in Boston with an additional office in Montreal. For more information, visit our website at Comlinkdata.com and follow us on Twitter and Instagram (@Comlinkdata) or LinkedIn: Comlinkdata.