Azure Data Engineer Job at TechnoGen, Raleigh, NC

VHk2M2VOSTJSR2IrdXJ6a3A3WHd3WXpLZVE9PQ==
  • TechnoGen
  • Raleigh, NC

Job Description

Position : Data Engineer Location : Remote Job Description : Experience working within the Azure ecosystem, including Azure AI Search, Azure Storage Blob, Azure Postgres and understanding how to leverage them for data processing, storage, and analytics tasks. Ability to preprocess and clean large datasets efficiently using Azure Tools /Python and other data manipulation tools. Experience with techniques such as data normalization, feature engineering, and data augmentation is preferred. Expertise in working with healthcare data standards (ex. HIPAA and FHIR), sensitive data and data masking techniques to mask personally identifiable information (PII) and protected health information (PHI) is essential. In-depth knowledge of search algorithms, indexing techniques, and retrieval models for effective information retrieval tasks. Familiarity with search platforms like Elasticsearch or Azure AI Search is a must. Familiarity with chunking techniques and working with vectors and vector databases like Pinecone. Ability to design, develop, and maintain scalable data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data. Experience with implementing best practices for data storage, retrieval, and access control to ensure data integrity, security, and compliance with regulatory requirements. Be able to implement efficient data processing workflows to support the training and evaluation of solutions using large language models, ensuring reliability, scalability, and performance. Ability to proactively identify and address issues related to data quality, pipeline failures, or resource contention, ensuring minimal disruption to systems. Experience with large language model frameworks, such as Langchain and know how to integrate them into data pipelines for natural language processing tasks. Experience working within the snowflake ecosystem. Knowledge of cloud computing principles and experience in deploying, scaling, and monitoring AI solutions on cloud platforms like Snowflake, Azure, AWS. Ability to communicate complex technical concepts effectively to technical and non-technical stakeholders and collaborate with cross-functional teams. Analytical mindset with a keen attention to detail, coupled with the ability to solve complex problems efficiently. Knowledge of cloud cost management principles and best practices to optimize cloud resource usage and minimize costs. Must Have: Minimum of 12 years' experience as a data engineer Hands-on experience with Azure Cloud eco-system. Hands-on experience using Python for data manipulation. Deep understanding of vectors and vector databases. Hands-on experience scaling POC to production. Hands-on experience using tools such as Document Intelligence (formerly Azure Form Recognizer), Snowflake, function app. Azure AI Search Experience working with PII/PHI Hands-on experience working with unstructured data. TechnoGen

Job Tags

Remote job,

Similar Jobs

Abode Care Partners

Complex Needs Medical Specialist (Hiring Immediately) Job at Abode Care Partners

 ...compensation structure, you will have the opportunity to grow your medical skills while providing personalized care to seniors and...  ...life and safeguarding the dignity of those we serve. For more information about Abode Care Partners, visit . Additional Job Information... 

Trilogy Health Services - Travel

Travel Certified Medication Aide Job at Trilogy Health Services - Travel

 ...Doing: Safely preparing, administering, and documenting all medications used in the health campus. Maintaining resident records regarding...  ...healthcare company in the Midwest. Our Certified Medication Aides are the staffing support system for Trilogy campuses throughout... 

LaSalle Network

Freelance Copywriter Job at LaSalle Network

LaSalle Network has partnered with a growing marketing agency looking to hire Freelance Copywriters in Q1 2025!Our client delivers strategic and creative solutions to complex marketing challenges, pushing the boundaries of what if to achieve real results. They are...

cewit

Remote Bookkeeper Job at cewit

 ...Strong mathematical and organizational skills. Ability to work independently and meet deadlines in a remote environment. Excellent...  ...to a supportive and collaborative team environment. Work-from-home setup, including a technology stipend if applicable.... 

Luzerne County, PA

Clerk Typist 3 (Local Government) - Luzerne County C&Y Job at Luzerne County, PA

Salary: $33,461.71 Annually Location : Wilkes-Barre, PA Job Type: Full-time Job Number: 01857 Department: Children & Youth Opening Date: 01/29/2025 Closing Date: 2/11/2025 11:59 PM Eastern Description This is a clerical position...