Data Engineer in New York, NY at Engage Partners, Inc.

Date Posted: 11/8/2020

Job Snapshot

  • Employee Type:
  • Location:
    New York, NY
  • Job Type:
  • Experience:
    Not Specified
  • Date Posted:

Job Description

Position Summary:
As a Data Engineer you will get to play a key and a collaborative role in the delivery of powerful data-driven products that support our mission of providing high-quality and low-cost healthcare to its union members. The Data Engineer will be responsible for providing internal analysts with accurate datasets by implementing best practices in data collection, movement, storage, and transformation of large datasets. This individual will work with both current ETL/Data Warehousing and provide direction for future development of data storage, streaming and pipeline architectures.
Essential Duties and Responsibilities:
- Work with Health Fund analytics, Operations and IT to implement new data infrastructure and migrate existing data from disparate sources into a robust data warehouse
- Using SQL, and other data transformation tools as well as languages such as Python and R, develop automated processes for transforming incoming data from multiple vendors and loading it into a centralized data warehouse for use by Health Fund Operations and Analytics
- Design, build, and manage automated test frameworks and scripts that support a continuous integration/continuous delivery (CI/CD) approach
- Generate subsets of data, and provide APIs and variables needed for interfacing with internal tools as well as public-facing websites
- Working with IT and Operations, to evaluate business cases for use of scalable cloud solutions such as Azure and Dynamics 365
- Interface with internal teams and vendors’ IT to work through data quality issues and champion HIPAA compliant best practices in data handling and transfer
- Create clear documentation (user guides, quick starts, schemas, and data dictionaries) of established data structures and use cases
- Provide tutorials and working sessions to empower analysts in creating queries and accessing right level of data and establish oneself as an expert resource on internal and external data sources and management
- Bachelor’s degree in Computer Science or a related discipline; Advanced degree preferred
- 5+ years of full-time experience or demonstrated accomplishments in relevant subject areas
- Prior experience in working with healthcare claims is a must
- Strong knowledge of SQL with hands-on industry experience is a must
- Knowledge of methods for handling non-relational JSON and XML formatted data
- Experience with document based NoSQL/Document storage databases (ex. Cosmos, Cassandra or MongoDB)
- Familiarity with database performance optimization techniques
- Experience with statistical packages such as STATA or SAS is preferred