Data Migration – Regulatory Services Systems

The Assignment

31ten was commissioned to migrate data from Breckland Council’s Ocella system to Idox Cloud for planning, building control and local land charges. We provided a full extract, transform and load (ETL) service providing Idox with the completed loader files to populate the new system.

We used our unique mix of skills: combining detailed knowledge of the services and software solutions, with technical database and data manipulation expertise to ensure the project ran smoothly.

Ocella was used by Breckland for planning, building control and local land charges; the council had procured a new system and commissioned 31ten to undertake a data migration strategy, this allowed us to undertake the necessary discovery and planning work which was essential to enable the practical work of data migration to take place. The strategy established the data structures in the legacy system, agreed the scope of the records to be migrated and the design of the technical data pipelines to enable the ETL to take place.

After we produced the strategy, we were recommissioned by the council to migrate the data from Ocella to Idox Cloud.
The project had multiple stakeholders including Breckland’s ICT team, the planning and building control team and the system supplier Idox.

The Role

We began the project by using the detailed discovery work that we’d established during the strategy phase to reconfirm the scope and approach of the migration for documents and data across the three services in question.

We produced, agreed and baselined the project plan which was a complex process as the sequencing needed to establish locked-in dates for multiple data and document loads across three services. Each data and document load needed its own and scope with key, specific outcomes planned with the council. This approach ensured maximum value was obtained from each data load.

The plan needed close alignment and sychronisation with the team who were testing and configuring the system across each data load for each service, so that the configuration of the system could be locked and baselined for each load, to ensure the data was extracted and loaded to a stable environment.

The final data loads needed particularly careful planning as the legacy system needed to stop being used entirely before the new system went live, furthermore the final load required the transferral of approximately one million documents which needed to align to the structured data.

Having established the tools and services which would build the data pipeline, we worked closely with Breckland’s ICT team to build the pipeline in the Breckland environment. This required tools to connect to the legacy Oracle database and a modern database environment to enable the transformation and validation of the extracted data. We used automated validation and worked carefully with the client to enable quality-assurance and automated checking of the transformed data before transferral.

Each data load was followed by the team at Breckland undertaking structured testing of the loaded data.

31ten’s data transformation allowed the optimal source to target mapping, and where there were underlying data quality issues, allowed us to help to provide an approach to improving the data quality in the target system.

 

Key Outcomes

  • Our work allowed the Idox Cloud system to successfully go live on time and within the allocated budget for the work.
  • This allowed Breckland to stop using and decommission their legacy Ocella system.
  • Our unique mix of skills and specialist council data migration service allowed the client to efficiently implement their new system, avoiding the costs and delays associated with repeated and unplanned data loads.