Diwali, the Hindu festival of lights, symbolizes the victory of light over darkness, good over evil, and knowledge over ignorance. While Diwali has a deep religious, social, and cultural significance, there are also precious lessons from the Diwali spirit that we can all learn as professionals.

Go Clean

Deep cleaning your home/physical spaces for Diwali is an age-old tradition. An annual cleaning exercise is done by the majority celebrating Diwali as cleanliness signifies letting go of what doesn’t work/serve you well and making space for the new.

Applying this thought to your professional work, think about what you can clean up from your career. This could be cleaning up the data in your phones, laptops, and other gadgets. This could also mean cleaning up your work profiles and resumes to represent the best of what you bring to the table in line with the promise of the future. You can also look at cleaning up your professional contacts/connections.

Let’s Upskill

As Diwali symbolizes knowledge over ignorance, this is a good time to invest in doing a critical skill audit of your work profile. Do take the time to look at what skills are relevant in the market based on your current profile and future aspirations. Next draw up a personal action plan for re-skilling or upskilling to be future-ready, have a competitive advantage, and ensure career advancement.

Say Yes to Networking

Diwali celebrations are about spending quality time with family and friends. Everyone makes the time to wish dear ones, catch up with close ones and re-connect with special people from the past.

Likewise, this is also a good time to build and nurture professional relationships. This can lead to new thinking and perspectives, access to valuable resources, career opportunities, and support systems that may help you navigate your professional journey better.

Dress to Impress

Another important aspect of Diwali is around dressing up. Typically, most people wear clean new and traditional attires. In addition, keeping up with the festive cheer, there is a lot of new age styling and experimentation in fashion!

Professionally, this can be an opportunity to re-look at your wardrobe and possibly invest in good business wear. This could be in terms of buying new clothes and styling existing clothes to make the right impression in work settings. While the definition of business attire can vary depending on the industry, company, and its culture, its significance is multifold –

• Confidence: Dressing professionally can boost self-confidence. When you feel good about how you look, you are likely to feel more self-assured, which can positively impact your performance.
• Productivity: Wearing appropriate professional attire can help you get into a professional mindset. If you stay in your pajamas all day, you are likely to feel lazy, similarly if you dress professionally, you’re likely to feel professional which might help you focus on your tasks more.
• Professional Growth: In many industries, dressing professionally is a part of the overall grooming and presentation that is expected for career growth. It can open doors to promotions and new opportunities. (The movie ‘The Devil Wears Prada’ is a great example of this.)

Exchange gifts 

Gifting is another important aspect of Diwali. As per tradition, it is customary to exchange sweets, dry fruits and/or home décor gifts to near and dear ones. This is a symbol of gratitude, love and appreciation for their presence and significance in your life.

In the work context, if you are wondering what to gift to your work colleagues, here are a few suggestions. You can gift a compliment to someone for work well done, gift a Thank you to someone for helping you, gift your time to someone who needs an ear to listen to their ideas/problems or stories, or even gift an online course to someone for their professional learning and development.

On that note, wishing you and your loved ones a very HAPPY DIWALI. We hope this Diwali you take the time to celebrate with family and friends as well as set aside some time to think about how you can sparkle your career with the festive cheer of Diwali!

The post Diwali Spirit To Light Up Your Career appeared first on Fresh Gravity.

The beginning of AI is shrouded with myths, stories, and rumors of artificial beings endowed with intelligence or consciousness by master craftsmen. One of the first formal beginnings of AI research was at a workshop held on the campus of Dartmouth College, USA during the summer of 1956. This was followed by an AI winter around 1974 when the US government withdrew all funding for AI research. This changed eventually when the Japanese government showcased major progress and heavily funded this field. The boom that we see today started in the first decade of the 21st century and of course, we are now at a point where AI impacts all areas of our lives and jobs. 

AI has been a hot topic for many decades now but its relevance for all data professionals is more today than it has ever been in the past. Recently, I had the opportunity to moderate a panel discussion hosted by Fresh Gravity on “Exploring the AI Frontier in Data Management for Data Professionals’. The panelists for the discussion, a group of talented data professionals with vast knowledge and in-depth experience, were Ayush Mittal (Manager, Data Science & Analytics – Fresh Gravity), Siddharth Mohanty (Sr Manager, Data Management – Fresh Gravity), Soumen Chakraborty (Director, Data Management – Fresh Gravity), and Vaibhav Sathe (Director, Data Management – Fresh Gravity). 

It was an opportunity to provide some thoughts and pointers on what we, as data professionals, should gear up on to be able to leverage various opportunities that AI-driven tools are providing and are expected to provide to enhance the value proposition we offer to our clients, help us perform our work smarter, and spend more time and effort on the right areas, instead of laboring over activities that can be done quicker and better by leveraging AI offerings. 

To summarize I would like to list some key take aways from this insightful session – 

• We all are experiencing the impact of AI in our everyday lives. The ability to harness and understand the nuances and be able to utilize the options (like personalized product suggestion in retail websites) can make our lives simpler, without allowing AI to control us
• Cybersecurity is a key concern for all of us. AI can detect and analyse complex patterns of malicious activity and quickly detect and respond to security threats
• Optimizing the use of AI can be a huge differentiator in delivering solutions for clients – where some of the tools that can be leveraged are StackOverflow, CoPilots, Google and AI driven data modelling tools
• For Data Management, with the huge volume and variety of data an organization has to deal with, the shift has already started. By using more AI-driven tools and services, organizations can ensure quicker insights, transformations, and movement of data across. This trend will only accelerate going forward
• AI will have a direct impact on improving the end user outcomes with speed of delivery and quality of data insights and predictions. What we see now is just the beginning of the huge shift in paradigm in the way value is delivered for the end user
• Establishing ethical usage of data and implementing Data Governance around data usage is key to AI success
• Everyone need not understand the code behind AI algorithms but should understand its core purpose and operational methodology. Truly harnessing the power of any AI system hinges on a blend of foundational knowledge and intuitive reasoning, ensuring its effective and optimal use
• Some upskilling and curiosity to learn are essential for each role (like Business Analysts, Quality Assurance Engineers, Data Analysts, etc.) to be able to take advantage of the AI-driven tools that are flooding the market and will continue to evolve
• While some of us may dream of getting back to an AI-less world, some are embracing the new AI-enabled world with glee! The reality is that AI is here to stay, and the way we approach and adapt to this revolution will determine whether we can benefit while staying within the boundaries of ethical limits

Fresh Gravity has rich experience and expertise in Artificial Intelligence. Our AI offerings include Deep Learning Solutions, Natural Language Processing (NLP) Services, Generative AI Solutions, and more. To learn more about how we can help elevate your data journey through AI, please write to us at info@freshgravity.com or you can directly reach out to me at Sudarsana.roychoudhury@freshgravity.com. 

The post Exploring the AI Frontier in Data Management for Data Professionals appeared first on Fresh Gravity.

In today’s data-driven world, organizations are relying more and more on data to make informed decisions. With the increasing volume, velocity, and variety of data, ensuring data quality has become a critical aspect of data management. However, as data pipelines become more complex and dynamic, traditional data quality practices are no longer enough. This is where data observability comes into play. In this blog post, we will explore what data observability is, why it is important, and how to implement it.

What is Data Observability?

Data observability is a set of practices that enable data teams to monitor and track the health and performance of their data pipelines in real time. This includes tracking metrics such as data completeness, accuracy, consistency, latency, throughput, and error rates. Data observability tools and platforms allow organizations to monitor and analyze data pipeline performance, identify, and resolve issues quickly, and improve the reliability and usefulness of their data.

The concept of data observability comes from the field of software engineering, where it is used to monitor and debug complex software systems. In data management, data observability is an extension of traditional data quality practices, with a greater emphasis on real-time monitoring and alerting. It is a proactive approach to data quality that focuses on identifying and addressing issues as they occur, rather than waiting until data quality problems are discovered downstream.

Why is Data Observability important?

Data observability is becoming increasingly important as organizations rely more on data to make critical decisions. With data pipelines becoming more complex and dynamic, ensuring data quality can be a challenging task. Traditional data quality practices, such as data profiling and data cleansing, are still important, but they are no longer sufficient.

Let’s consider an example to understand why data observability is needed over traditional data quality practices. Imagine a company that relies on a data pipeline to process and analyze customer data. The data pipeline consists of multiple stages: extraction, transformation, and loading into a data warehouse. The company has implemented traditional data quality practices, such as data profiling and data cleansing, to ensure data quality.

However, one day the company’s marketing team notices that some of the customer data is missing in their analysis. The team investigates and discovers that the data pipeline had a connectivity issue, which caused some data to be dropped during the transformation stage. The traditional data quality practices did not catch this issue, as they only checked the data after it was loaded into the data warehouse.

With data observability, the company could have detected the connectivity issue in real time and fixed it before any data was lost. By monitoring data pipeline performance in real-time, data observability can help organizations identify and resolve issues quickly, reducing the risk of data-related errors and improving overall data pipeline performance.

In this example, traditional data quality practices were not sufficient to detect the connectivity issue, highlighting the importance of implementing data observability to ensure the health and performance of data pipelines.

Data observability provides organizations with real-time insights into the health and performance of their data pipelines. This allows organizations to identify and resolve issues quickly, reducing the risk of data-related errors and improving the reliability and usefulness of their data. With data observability, organizations can make more informed decisions based on high-quality data.

How to Implement Data Observability ?

Implementing data observability requires a combination of technology and process changes. Here are some key steps to follow:

Define Metrics: Start by defining the metrics that you want to track. This could include metrics related to data quality, such as completeness, accuracy, and consistency, as well as metrics related to data pipeline performance, such as throughput, latency, and error rates.

Choose Tools: Choose the right tools to help you monitor and track these metrics. This could include data quality tools, monitoring tools, or observability platforms.

Monitor Data: Use these tools to monitor the behavior and performance of data pipelines in real time. This will help you to identify and resolve issues quickly.

Analyze Data: Analyze the data that you are collecting to identify trends and patterns. This can help you to identify potential issues before they become problems.

Act: Finally, take action based on the insights that you have gained from your monitoring and analysis. This could include making changes to your data pipeline or addressing issues with specific data sources.

Benefits of Data Observability

Implementing data observability provides numerous benefits, including:

Improved Data Quality: By monitoring data pipeline performance in real time, organizations can quickly identify and address data quality issues, improving the reliability and usefulness of their data.

Faster Issue Resolution: With real-time monitoring and alerting, organizations can identify and resolve data pipeline issues quickly, reducing the risk of data-related errors and improving overall data pipeline performance.

Better Decision Making: With high-quality data, organizations can make more informed decisions, leading to improved business outcomes.

Increased Efficiency: By identifying and addressing data pipeline issues quickly, organizations can reduce the time and effort required to manage data pipelines, increasing overall efficiency.

Data observability is a new concept that is becoming increasingly important in the field of data management. By providing real-time monitoring and alerting of data pipelines, data observability can help to ensure the quality, reliability, and usefulness of data. Implementing data observability requires a combination of technology and process changes, but the benefits are significant and can help organizations to make better decisions based on high-quality data.

The post Data Observability is the new Data Quality – What, Why and How ? appeared first on Fresh Gravity.

In my opinion, a coder can be guilty of two things. Either we over-engineer i.e., try to solve everything in one go, instead of following an iterative approach, OR we under-engineer, i.e., just code without understanding the impact. What we need is to attain the ‘middle ground’.

Here are 7 guidelines to ensure we are always in the ‘middle ground’:

1) Don’t just code the requirement. You must understand the problem fully. You’re a Data Person, you should care about the problem from the data’s perspective. Building complex code is cool, but spend more time understanding and analyzing the requirements from the data’s point of view. That is more important than what tool, language, or technology you are using to process it.

2) Unit Testing is part of coding, not a separate exercise. Dedicate 25-30% of development time to Unit Testing. As an example, if it takes 8 hours to code, you should allocate a minimum of 2 hours to Unit Test that code. In my opinion, 75-85% of testing coverage should come from Unit Testing and the rest from Test Automation. Remember, SIT (System Integration Testing) is not for testing one piece of the puzzle (your code only) but the entire puzzle board (all integrated code). So don’t rely on your best friend on the QA (Quality Assurance) team to figure out what you did last summer. Spend time thinking about the test cases, for example, a data integrity check before and after processing, or code performance metrics. If you are NOT clear on the unit test cases, then don’t start coding. Seek more clarity until you can visualize the output. Keep in mind, that unit testing is not just checking if the code runs but checking if it generates the right output in the specified time.

3) Don’t use a hammer to crack a nut. You don’t need to consider all possible edge cases in the world while designing. Perfect code that sits in your machine has no impact, whereas merely “okay” code in production adds value. Keep your design simple but ensure the code is nimble; you can always increase complexity later if needed. Question the design. One of the most common reasons for poor design is NOT understanding the underlying technology enough and trying to solve every need with a custom approach. For example: if you have more supporting custom tables to hold your code processing information than actual data tables, then you are either not using the right tools for processing, OR not using the out-of-the-box features efficiently. This design is neither sustainable nor scalable.

4) Don’t let your experience take over your imagination. Very often we refuse to see the problem with fresh eyes and always try to tie every new problem back to problems we have solved earlier. That’s the wrong approach. Keep in mind, we are living in an age where technological advancement occurs rapidly. Do your due diligence and see what’s new before dusting off your old toolbox.

5) Asking for help and using Google (now ChatGPT), is the most powerful skill. There is no point in spending days trying to solve a problem yourself when someone has already done it or can do it for you within minutes. However, before asking for help, document the logical steps you’ve followed with pseudo code and summarize why you think it’s not working. This logical breakdown not only helps an expert to make a resolution faster but also helps you search for the right content.

6) Reusability is the key. Make sure your code is well documented, clearly comment on your code, make it modular (break your code into logical units that can be tested individually), and make it configuration-driven. Anyone (including you) should be able to easily understand (remember) what you did a few months or even years ago.

7) GIT is your best friend, NOT some annoying Ex from your past. So, please stop treating GIT as an “extra” task! Once you make using GIT a habit you will realize how it makes your life easier. Follow some basic rules of thumb: Take a feature branch approach, always pull before push, push daily (and encourage others to do the same), merge feature with dev only after the feature is tested, and do not push to master. Trust me on this, you will thank me later. Code Repos are invented to help developers, not the other way around.

In the end, it’s all about having fun. Keep in mind, that the code you write, whether small or big, easy, or complex, is your unique creation. It’s your legacy, so treat it well. Otherwise, what’s the point?

If you have any thoughts, comments, ideas, or feedback, please reach out at soumen.chakraborty@freshgravity.com.

The post A Coder’s Legacy: 7 Guidelines if you work in the Data Management space appeared first on Fresh Gravity.

Data is one of the most significant assets for any organization and those who are able to effectively collect, analyze, and make data-driven decisions stand to have a significant advantage over their competitors. Therefore, trusting that data is paramount to success.  

Gartner predicts that by 2025, 60% of data quality monitoring processes will be autonomously embedded and integrated into critical business workflows. Even with all the advanced technologies around, currently, this process is still 50-70% manual, as it follows a reactive approach. It solely depends on Data Subject Matter Experts (SMEs)/Stewards; so instead of focusing 100% on data insights, the bulk of their time goes into constant sampling, profiling, and adding new data monitoring rules to ensure the data is accurate, complete, consistent, and unique. To determine the health of the systems, these types of data monitoring necessitate data SMEs tracking pre-defined metrics; which essentially means, they must know what issues they are looking for and what information to track first. With this reactive approach, only 25-40% of Data Quality (DQ) problems get identified before they create a trickle-down impact. Hence, organizations need a proactive data health monitoring approach where data observability on top of data quality will come into play.  

Bringing Data Quality and Observability together, here’s the ultimate solution to achieving healthy data: 

Even though data observability is built on the concept of data quality, it goes beyond that by not just describing the problem but by explaining (even resolving it) and preventing it from recurring in the future.  

Data-driven organizations should focus on the following five pillars to provide real-time insights into data quality and reliability, along with the traditional data quality dimensions: 

• Freshness: Check how current the data is and how often the data is updated 
• Distribution: Check if the data values fall within an acceptable range; reject or alert if not 
• Volume: Check if the data is complete and consistent; identify the root cause if not and provide recommendations 
• Schema: Track changes in data organization that give real-time updates of changes made by multiple contributors
• Lineage: Record and document the entire flow of data from initial sources to end consumption 

By observing these five pillars, data SMEs (Subject Matter Experts) can gain new insights into how data interacts with different tools and moves around their IT infrastructure.  

This will help find issues and/or improvements that were not anticipated, resulting in a faster mean time to detection (MTTD) and mean time to resolution (MTTR). However, this is easier said than done. This is because the current technology landscape does not have many tools that can proactively observe data based on these five pillars.  

The Future of Data Quality is to be proactive. 

Pro–active monitoring is a key component to gaining more value from data. By proactively monitoring observability and quality, organizations can identify issues quickly and resolve them before they become major problems. This will also help in understanding the data better, resulting in better decision-making and improved customer experiences. 

This is where Fresh Gravity’s DOMaQ tool holds its niche in enabling the business as well as technical users to identify, analyze, and resolve data quality and data observability issues. The DOMaQ tool uses a mature AI-driven prediction engine. 

Fresh Gravity’s DOMaQ Tool 

Fresh Gravity’s DOMaQ (Data Observability, Monitoring, and Data Quality Engine) enables business users, data analysts, data engineers, and data architects to detect, predict, prevent, and resolve issues, sometimes in an automated fashion, that would otherwise break production analytics and AI. It takes the load off the enterprise data team by ensuring that the data is constantly monitored, data anomalies are automatically detected, and future data issues are proactively predicted without any manual intervention. This comprehensive data observability, monitoring, and data quality tool is built to ensure optimum scalability and uses AI/ML algorithms extensively for accuracy and efficiency. DOMaQ proves to be a game-changer when used in conjunction with an enterprise’s data management projects (MDM, Data Lake, and Data Warehouse Implementations).   

Key Features of Fresh Gravity’s DOMaQ tool: 

• Connects, scans, and inspects data from a wide range of sources
• Saves 50-70% of arduous manual labor through auto-profiling 
• Automates data quality controls by using machine learning to explain the root cause of the problem and predicts new monitoring rules based on evolving data patterns and sources 
• Comes with 100+ data quality/validation rules to monitor the consistency, completeness, correctness, and uniqueness of data periodically or constantly  
• Helps in preventing trickle-down impact by generating alerts when data quality deteriorates 
• Supports collaborative workflows. Users can keep their work in a segregated manner or can share among the team for review/reusability 
• Allows users to generate reports, build data quality KPIs, and share data health status across the organization

The future of data quality with DOMaQ is magical since this AI-driven proactive monitoring will enable businesses and IT to work together on the data from inception to consumption and will ensure that the “data can be trusted.” 

To learn more about the tool, click here.

If you’d like a demo, please write to vidula.kalsekar@freshgravity.com or soumen.chakraborty@freshgravity.com. 

The post The Power of Pro-active Monitoring: Why Data Observability and Data Quality Matter appeared first on Fresh Gravity.

This is the decade for data transformation. The key is to ensure that data is available for driving critical business decisions. The capabilities that an organization will absolutely need are: 

• Data as a product where teams can access the data instantly and securely 
• Data sovereignty is where the governance policies and processes are well understood and implemented by a combination of people, processes, and tools
• Data as a prized asset in the organization – Data maintenance with high quality, consistency, and trust 

The significance of data to run an enterprise business is critical. Data is recognized to be a major factor in driving informed business decisions and providing optimal service to clients. Hence, it is crucial to have good-quality data and an optimized approach to DQM. 

As the volume of data in an organization increases exponentially, manual DQM becomes challenging. Data that flows into an organization is of high volume, mostly real-time and may change characteristics. Advanced tools and modern technology are needed to provide the automation that would drive accuracy and speed to achieve the desired level of data quality for an organization.  

DQM in an enterprise is a continuous journey. To be relevant for the business, a regular flow of learning needs to be fed into the process. This is important for improving the results as well as for adapting to the changing nature of data in the enterprise. A continuous process of assessment of data quality, implementation of rules, data remediation, and learning feedback is necessary to run a successful DQM program.  

The process of DQM can be depicted with the help of the following diagram – 

How does Machine Learning (ML) help in DQM? 

To drive these capabilities and accelerate data transformation for an organization, it is extremely important to have a strong DQM strategy. The burden of DQM needs to shift from manual mode to a more agile, scalable and automated process. The time-to-value for an organization’s DQM investments should be minimized. 

Focusing on innovation, not infrastructure, is how businesses can get value from data and differentiate themselves from their competitors. More than ever, time, and how it’s spent, is perhaps a company’s most valuable asset. Business and IT teams today need to be spending their time driving innovation, and not spending hours on manual tasks.  

By taking over DQM tasks that have traditionally been more manual, an ML-driven solution can streamline the process in an efficient cost-effective manner. Since an ML-based solution can learn more about an organization and its data, it can make more intelligent decisions about the data it manages, with minimal human intervention. The nature of data in an organization is also ever-changing. DQM rules need to constantly adapt to such changes. ML-driven automation can be applied to greatly automate and enhance all the dimensions of Data Quality, to ensure speed and accuracy for small to gigantic data volumes.  

The application of ML in different DQ dimensions can be articulated as below: 

• Accuracy: Automated data correction based on business rules
• Completeness: Automated addition of missing values  
• Consistency: Delivery of consistent data across the organization without manual errors 
• Timeliness: Ingesting and federating large volumes of data at scale  
• Validity: Flagging inaccurate data based on business rules 
• Uniqueness: Matching data with existing data sets, and removing duplicate data

Fresh Gravity’s Approach to Data Quality Management 

Our team has a deep and varied experience in Data Management and comes with an efficient and innovative approach to effectively help in an organization’s DQM process. Fresh Gravity can help with defining the right strategy and roadmap to achieve an organization’s Data Transformation goals.  

One of the solutions that we have developed at Fresh Gravity is DOMaQ (Data Observability, Monitoring, and Data Quality Engine), which enables business users, data analysts, data engineers, and data architects to detect, predict, prevent, and resolve issues, sometimes in an automated fashion, that would otherwise break production analytics and AI. It takes the load off the enterprise data team by ensuring that the data is constantly monitored, data anomalies are automatically detected, and future data issues are proactively predicted without any manual intervention. This comprehensive data observability, monitoring, and data quality tool is built to ensure optimum scalability and uses AI/ML algorithms extensively for accuracy and efficiency. DOMaQ proves to be a game-changer when used in conjunction with an enterprise’s data management projects (MDM, Data Lake, and Data Warehouse Implementations).

To learn more about the tool, click here.

For a demo of the tool or for more information about Fresh Gravity’s approach to Data Quality Management, please write to us at soumen.chakraborty@freshgravity.com, vaibhav.sathe@freshgravity.com or sudarsana.roychoudhury@freshgravity.com.

The post Optimizing Data Quality Management (DQM) appeared first on Fresh Gravity.

What is IDMP?

If you work in the Life Sciences industry, the term “IDMP” would be familiar to you.  IDMP stands for Identification of Medicinal Products which is a set of standards used to globally standardize data and structures to define and uniquely identify medicinal products. IDMP comprises of a set of five standards developed by ISO, or the International Organization for Standardization.

The five ISO IDMP standards, all of which deal with unique identification and exchange of information for medicinal products, are:

• ISO 11615: Standards relating to Medicinal Product
• ISO 11616: Standards relating to Pharmaceutical Product
• ISO 11238: Standards relating to Substances.
• ISO 11239: Standards relating to pharmaceutical dose forms, units of presentation, routes of administration, and packaging items related to medicinal products.
• ISO 11240: Standards relating to units of measurement.

The 5 ISO standards are illustrated and described in Figure 1:

Why IDMP?

The competitive, dynamic, and highly governed nature of the Life Sciences industry requires a continuous flow and exchange of data between regulatory authorities, pharmaceutical companies, and manufacturers, among other stakeholders. While there has always been a need for efficiently managing submissions and adverse events reporting, there lacked a single mechanism to reliably exchange accurate information between stakeholders. To address this need in the context of pharmacovigilance and improving adverse event reporting, IDMP was developed.

ISO IDMP provides a standard way for defining and storing medicinal product information, which will enable efficient reporting, tracking, faster decision-making, and response during adverse event reporting. At present IDMP focuses on standardization of data with a future goal of improving overall pharmacovigilance[1].

Currently, most of the medicinal product information is spread across fragmented systems within pharmaceutical organizations. Some of the systems are legacy systems and hold data in an unstructured format such as documents, pdfs, excel workbooks, and email messages. Standardizing this data in the IDMP format supports the regulatory submission processes within an organization as it not only maintains uniformity in managing data assets within an organization but also in exchanging data between the regulators (such as the EMA and the FDA) and Life Sciences organizations. The Medicinal Product data previously submitted to regulatory authorities can be re-used when submitting variations to authorized medicinal products. In case of additional requests from regulatory authorities for a submission or a query, the information will be readily accessible. For an Investigational Medicinal Product, as the clinical trial progresses, data generated through different business processes can be submitted periodically and standardization in IDMP format will help maintain transparency.

What has happened so far?

The European Medicines Agency (EMA), a regulatory agency of the European Union (EU) is pioneering the IDMP journey and has organized the implementation using its SPOR services. SPOR is a set of four data management services, namely:

• Substance Management Service (SMS)
• Product Management Service (PMS)
• Organization Management Service (OMS)
• Referential Management Service (RMS)

SPOR covers multiple master data domains of the medicinal products definition. IDMP is being implemented by EMA in phases with different timelines. RMS and OMS services were launched in 2017 and are currently used in submissions that need to be made to the EMA and other regulatory authorities within Europe. As per timelines published by EMA, the next in line is the implementation of the Product Management Service (PMS), expected to go live in Q2 2023. This means that all submissions for medicinal products will need to be made to the PMS service using web-based Digital Application Dataset Integration (DADI) forms (Latest Implementation Guidance IG v2.1 published here). Regulatory organizations, just like the EMA, will be holding the beacon for guidance on IDMP compliance for the industry.

The United States Food and Drug Administration (FDA) has not yet mandated the use of IDMP standards for the submission of data to the FDA. Nevertheless, preparation for EMAs SPOR services will also ensure readiness for FDAs’ future requirements.

Figure 2 is an illustrative depiction of the different approaches undertaken by the US FDA and EMA for defining IDMP ISO standards.

What next?

As IDMP timelines for EMA draw near, Life Sciences organizations are embarking on a digital transformation journey. At a strategy level, an organization can lay down a roadmap for IDMP and align it with its long-term goals to build Enterprise level Data Assets. IDMP compliance is to be viewed as an opportunity to break organizational data silos, improve overall organizational data quality and governance, and enhance operational efficiencies.

Organizations will have to be agile to implement IDMP. Life Sciences organizations and regulators will have to work in collaboration to fully leverage the adaptive, scalable, and nimble technologies available in the market, to achieve 100 percent IDMP compliance.

Did you know about our IDMP solution?

Fresh Gravity has built an MDM-driven solution to address IDMP Compliance needs. Read more about Fresh Gravity’s approach to this here.

For a demo of Fresh Gravity’s solution, or more information and questions, please write to neha.inamdar@freshgravity.com.

For more detailed information about this solution, please refer to the datasheet here.

[1] Pharmacovigilance is the science and activities relating to the detection, assessment, understanding, and prevention of adverse effects or any other medicine/vaccine-related problem. [Source:who.int].

The post Demystifying IDMP – All you need to know to get started! appeared first on Fresh Gravity.