European Weather Data Migration

Last updated on 


Introducing the European Weather Data Migration

1. Check raw data

A process to remove characters from time values seems necessary.

2. Set the Preprocessing Plan

  • Data Cleaning.
    • Remove characters from time values.
    • Rename time column.
  • Convert to the format: Name, Time, Value & Convert to UTC time.

Data Cleaning

  • Remove characters from time values.
  • Rename time column.

Convert to the format: Name, Time, Value & Convert to UTC time

Once the data frame is restructured as shown in the image below, it will be ready for upload to Machbase Neo.

3. Data Upload

Finally, the data can be uploaded to Machbase Neo using the command below.

machbase-neo shell import --input ./datahub-2025-2-EU-weather.csv.gz --compress gzip --header --method append --timeformat ns eu_weather

Check the entire code.

datahub/dataset/2025/02.European Weather Data/conv/convert.py at main · machbase/datahub
All Industrial IoT DataHub with data visualization and AI source - machbase/datahub
GitHubmachbase

4. Check the results after uploading

Output when executing the following code in the Machbase Neo internal shell.

select * from v$eu_weather_stat;

※ Subsequent link to the AI training process: European Weather Data

Back to Top