Investment Management · Data & CloudCase 001
From valuable prototype to reliable data platform.
A lean investment team built a sophisticated internal data platform that became critical to its daily research process. Its success created a new problem: the same small team benefiting from the system was increasingly responsible for maintaining the infrastructure underneath it.
Engagement in progress
What started as a prototype became a system the firm depended on every day.
The platform began as a practical solution built by technically capable members of the firm alongside their other responsibilities. It automated data ingestion, centralized research data, and made large datasets accessible for analysis. As its value grew, it quietly became production infrastructure.
01The challenge
Success created operational burden.
The platform worked. The challenge was keeping it working.
Storage systems needed attention. Query infrastructure had to remain healthy. Data jobs needed monitoring. Software required upgrades. Failures needed investigation. Observability, backups, networking, and access controls all became recurring responsibilities.
None of those tasks were unusual on their own. The problem was accumulation.
The same lean team responsible for research, engineering, and investment technology was increasingly responsible for operating the infrastructure underneath it.
Every hour spent maintaining the platform was an hour not spent benefiting from it.
The starting point
- Data Sources
- Custom Ingestion
- Self-Managed Data Platform
- Research & Analysis
Small internal team maintains every layer
02The approach
Keep what creates value. Remove what creates unnecessary work.
Salcorsa began by separating the technology that differentiated the firm’s research process from infrastructure the team was operating simply because someone once had to build it.
Differentiating
The firm’s data, ingestion logic, research workflows, models, and proprietary systems represented real intellectual property.
Commodity
Operating commodity storage, query infrastructure, catalog services, monitoring systems, and supporting platform components did not.
Keep ownership of what creates investment value. Stop owning infrastructure just because you can.
Rather than rebuild the platform, Salcorsa designed an incremental modernization that preserved working data pipelines and research workflows while moving appropriate infrastructure onto managed cloud services.
Before & after
Self-managed platform
- Data Sources
- Custom Ingestion Team-operated
- Self-Managed Infrastructure Team-operated
- Research & Analysis
The internal team operates the platform and uses the platform.
Managed platform
- Data Sources
- Existing Ingestion Team-owned
- Managed Cloud Infrastructure Managed service
- Research & Analysis
The internal team focuses on using and improving the system.
03The solution
Modernize underneath the workflows.
The existing platform was technically sophisticated and built with modern open-source technologies. Salcorsa’s goal was not to discard that work.
Existing ingestion and research workflows are being preserved wherever practical while the storage, analytical, catalog, observability, and supporting platform layers move toward managed services.
The migration is intentionally incremental, reducing risk and avoiding an infrastructure project becoming an unnecessary rewrite of working software.
04The impact
Less infrastructure. More leverage.
The important outcome is not simply moving workloads to the cloud. It is changing what the team has to think about every day.
Before
- Is the storage system healthy?
- Why did the query service fail?
- Did the upgrade break something?
- Who is fixing the ingestion issue?
After
- What can we learn from this dataset?
- How can we improve the research process?
- What should we automate next?
- What should we build next?
The team can spend more time benefiting from the platform and less time maintaining the platform.
Results
Measuring the operational difference.
As the migration progresses, Salcorsa is measuring the reduction in self-managed services, recurring maintenance effort, platform incidents, infrastructure cost, data reliability, and migration impact.
Self-managed services eliminated
Engineering hours returned
Scheduled data reliability
Infrastructure cost
Make success sustainable.
Lean teams are often capable of building remarkably sophisticated systems. The challenge comes when every successful internal system also creates another production platform the organization must operate indefinitely.
Salcorsa helps teams preserve the technology that differentiates their business while reducing the operational machinery surrounding it.
The goal wasn’t to replace what the team built. It was to make its success sustainable.
Next step→
Have a system that has become harder to operate than it should be?
This case study describes work performed for an affiliated investment technology environment.