Co-authored by Elizabeth Press and Vincenzo Porpiglia
You can listen to the podcast on Spotify here
In recent years, there has been a growing interest in the concept of a data mesh within the technology industry. Data mesh is a sociotechnical approach to build a decentralized data architecture by leveraging a domain-oriented, self-serve design, according to Wikipedia.
Many organizations hope to harness the Data Mesh to build an organization that enables data to be easily accessible, discoverable, and secure. This concept has been gaining popularity as companies seek to scale their data operations, leverage domain expertise to deliver more value from analytics and drive a more product-focused approach to data.
Adoption of the Data Mesh is often a long-term, costly process that necessitates strategic thought, planning and continuity.
Three of the most important steps for planning and implementing a Data Mesh include:
Overcoming the us against them mentality: Data against the business
The biggest gap in the industry is data’s over-focus on tech rather than business goals. Many data professionals within organizations can have an „us against business“ attitude that is not productive, leading to disconnection and discord between data and other functions. Because of the data mesh’s domain-ownership orientation, organizations are exploring the adoption of a data mesh. The Data Mesh concept allows data colleagues to work as a team towards a common goal and be embedded within domain organizations.
Professionalizing the existing data mess: DataOps, Product Management and other methods.
To prevent a data mesh from turning into a data mess, there are several key steps that organizations need to take. Firstly, governance is key. This means having a well-defined DataOps process that brings together data engineering, engineering, product, QA, privacy & information security, and data product managers to work as a team. Secondly, product management is critical. The self-service products, data schema and other elements need to be managed like any other web-based product, with an understandable UX and consistent data structure and definitions throughout the company. Thirdly, there needs to be a culture of documentation, with time budgeted for documenting the data mesh. Finally, data needs to be easily discoverable and accessible, with privacy, compliance and information security proactively managed by design.
Correct road mapping and budgeting for the Data Mesh
Implementing a data mesh can be expensive, but the long-term benefits are worth the investment. At first, implementing a data mesh will cost more, with investments needed in vanquishing technical debt, implementing data catalogs, documenting processes, and necessary migrations. However, in the long run, the ROI on data will be higher, with less time spent building and fixing pipelines, less duplicated data sets, and less friction between data and stakeholder teams.
In conclusion, the adoption of a data mesh offers many benefits for organizations looking to scale their data operations, deliver more value from analytics and drive a more product-focused approach to data. By overcoming the biggest hurdles of culture and business architecture, and by taking the steps necessary to prevent a data mesh from turning into a data mess, organizations can reap the benefits of a cost-efficient data mesh for years to come.
