Click to learn more about author Einat Orr. Data engineering is the science and art of producing good and timely data. Its goal is to deliver data to users even more than to deliver applications. There are great methods and tools that help deliver applications with consistently high quality. What are the methods and tools […]
How to Avoid the Not So Mythical $50,000 Query in the Cloud
Click to learn more about author Chris Lynch. So, you’ve made the decision to go all-in on the cloud. (BTW, I don’t blame you — as a CEO, I’ve made the same choice myself, both for the company I run and the companies that I invest in.) Now your attention needs to focus on how […]
The Myth of “The Cloud is Cheaper”
Click to learn more about author Chris Lynch. As we have all experienced recently, change is the only constant that continues to be true, decade in and decade out. Irrespective of pandemics, macro or microeconomic uncertainty, or even geopolitical unrest — one thing is guaranteed, especially in tech: change. There’s been a significant investment of […]
Data Architect vs. Data Modeler vs. Data Engineer
Michael Bowers, author and Chief Data Architect at FairCom Corporation, initially set out to research three careers in his presentation titled Data Architect vs. Data Modeler vs. Data Engineer for the DATAVERSITY® Data Architecture Online 2019 Conference. The process brought him to a wealth of information he would have appreciated much earlier in his career, […]
Ascend.io Expands Its Unified Data Engineering Platform
A new press release states, “Ascend.io, the data engineering company, today announced it has expanded the Ascend Unified Data Engineering Platform with the addition of Ascend Govern, a first of its kind suite of tracking, reporting, and security capabilities for a more granular understanding of how data is being used throughout an organization. Ascend Govern […]
The Chief Data Officer and the Chief Digital Officer: Work Together, Not Apart
Data vs. digital: That’s a big tension within many organizations. Chief Data Officer s and Chief Digital Officers don’t always agree about some important things, said Joe Caserta, president of consulting firm Caserta, during his DATAVERSITY® Enterprise Data World Conference presentation titled Building a Foundation for Disruption and Advanced Analytics. What’s the disconnect between the […]
Case Study: Deriving Spark Encoders and Schemas Using Implicits
Click to learn more about author Dávid Szakallas. In recent years, the size and complexity of our Identity Graph, a data lake containing identity information about people and businesses around the world, begged the addition of Big Data technologies in the ingestion process. We used Apache Pig initially, and then migrated to Apache Spark a […]
Credulous Data Science: Part Two – Surveys
Click to learn more about author Steve Miller. This is the Second Part of a two-part series. Read Part One here. About ten years ago, a software vendor with whom my intelligence consulting company partnered conducted a survey on the usage of various analytics platforms that were then prominent in the space. As a proselyte […]
Simplistic and Credulous Data Science: Part One
Click to learn more about author Steve Miller. A few months ago, I was solicited by one of my many LinkedIn Data Science groups to participate in a survey. I’m generally a bit suspicious of surveys, reviewing the questions carefully before making the decision to respond. This initiative, though, posed a question that quite intrigued […]
Building the Future of Women in Data
Careers involving data expertise are soaring. Data skills have seen significant growth in recent years, according to The Quant Crunch report. By 2020, the report predicts, the number of positions for data and analytics talent in the United States will increase by 364,000 openings, to 2,720,000. Clinical Data Analysis (+54 percent) Data Science (+40 […]