Analytics Podcast Episodes

Software engineering is often more art than science, making it difficult to measure productivity. There are ways to use data to be more effective as an individual contributor or an engineering leader, but surprisingly, engineering organizations and teams typically are not data-driven.

MergeStat is on a mission to change this with open-source, operational analytics for software engineering organizations. MergeStat started as an experiment to bring together two technologies: SQL and Git repositories. MergeStat provides data integration for your Git repositories, facilitating the exploration of legacy code and identification of code that hadn't been touched in a while and maybe deserved new attention.

From there, the use cases evolved. Today, MergeStat is used by organizations that have hundreds or even thousands of repositories. MergeStat is data infrastructure for Git repositories, where anyone can query the history and contents of their code bases.

Behind the scenes, MergeStat syncs data from the tools used to build and ship software into a PostgreSQL instance, as APIs provided by these tools are not always easy to understand and extract data from. MergeStat puts a lot of the usual work into implementing good API data consumption, like pagination and respecting rate limits.

From there, a user can query their data directly in MergeStat, or use other business intelligence tools and dashboards that know how to speak to PostgreSQL. See this example Grafana dashboard for GitHub pull requests.

Patrick DeVivo is Founder and CEO at MergeStat. In this session, we start out with a general overview of MergeStat and how it's used today.

Patrick explains how MergeStat is a general-purpose engine that companies use to craft the queries that fit their organization. We go into a few MergeStat use cases that Patrick sees today:

• In some cases, the actual data collection is the use case. For example, with audits the action is to deliver the list of pull requests that didn't follow best practices.
• Understanding the different versions of a programming language in use. If you're a Go shop, a single query aggregates the different Go versions used across all repositories.
• Find pull requests that have been open for a long time or merged without review.

Patrick's advice is to use MergeStat in a way that is positive and constructive to take action. Watch this episode to learn more about data integration for the software development lifecycle.

Dvir Mizrahi is Head of Financial Engineering at Wix, the leader in website creation with 220 million users running e-commerce operations. And with over six thousand employees, Wix ships more than fifty thousand builds each day.

Dvir is also among the original authors of the AWS Cloud Financial Management certification.

In this episode, Dvir covers how Wix shifted from FinOps to Financial Engineering. It's an engineering-first approach to build tooling and processes tracking financial key performance indicators (KPIs) for its multi-cloud infrastructure. The new approach established a culture of financial responsibility that supports Wix's continued growth.

Wix started in 2006 and initially ran its infrastructure on-premise. Today, Wix runs a multi-cloud environment on Google Cloud and Amazon Web Services (AWS). As Wix shifted from on-premise to the cloud, the procurement process of resources changed with it.

In the old world, purchasing additional hardware was a closed and controlled process. But in the cloud, Dvir compares resource procurement to "a supermarket where people can go in, take whatever they want, and leave without passing the registers." A developer could spin up a hundred thousand instances with just the click of a button.

Wix realized the financial risk that comes with liberal permissions to spin up infrastructure and hired Dvir in 2017. FinOps approaches infrastructure governance from a billing perspective and handles workloads already provisioned in the cloud. But at Wix's scale, where there are thousands of engineers, the FinOps approach stops working. "By the time you have a financial incident, it's too late and you didn't govern anything."

Dvir shifted the strategy to proactively preventing waste in the first place, by incorporating financial KPIs into engineering goals. In addition, Dvir built an internal platform called "InfraGod" which collects infrastructure data, integrates with Terraform, and enforces rules at the time of resource provisioning. Taking action at the time resources are provisioned rather than after the fact is "the difference between Finance and Financial Engineering."

Listen to this episode for a deep dive into the tactics that Dvir uses to run Financial Engineering at Wix, such as data collection, engineering post-mortems, monthly reports, and mandatory resource tagging.

Historically, the distribution and usage of open-source projects have been challenging to measure. 📏

Understanding your user base is critical to product planning and development. However, open-source maintainers have resorted to inelegant tactics to gather data on their users—such as scraping GitHub user data, performing reverse IP lookups on website traffic, or simply hoping that users submit support requests.

Scarf aims to solve this problem with its Scarf Gateway and Documentation Insights. The Scarf Gateway provides distribution analytics for open-source software and helps maintainers connect with commercial users. Documentation Insights aid in understanding how users interact with project websites and documentation.

In this episode, Lars chats with Avi Press, Founder and CEO at Scarf. Avi shares how Scarf grew from a hobby project into a venture-funded startup, as well as his thoughts on the future of open-source business models.

Companies build in the cloud for growth and speed. 📈

Engineering teams love building new things—so much so that cloud spend commonly becomes a major part of a company's profit and loss statement (P&L).

Cloud vendors have introduced pricing and discounting schemes to incentivize increased consumption and lock in long-term commitments from customers. Management gets involved at this point, but they often lack context and understanding of how cloud procurement works.

Forecasting cloud spend and aligning growth with infrastructure efficiency become important capabilities when you are about to sign a multi-million three-year contract with a cloud vendor. 💰

In this episode, Lars talks with Dieter Matzion, Senior Cloud Governance Engineer at Roku and long-time expert in cloud procurement and cloud financial operations. Before joining Roku, Dieter was an engineer at Google, Netflix, and Intuit, where he established infrastructure efficiency programs that combined cloud operations, analytics, and finance.