Kaggle Smart Home Data Migration



Introducing the Kaggle Smart Home Data Migration

1. Check raw data

2. Set the Preprocessing Plan

Since the time type is already in UTC, only the following preprocessing steps will be performed

  • Remove missing values.
  • Remove unnecessary columns.
  • Convert to the format: Name, Time, Value.

Remove missing values

df = df.dropna()
Remove unnecessary columns

Remove the columns 'icon', 'summary', and 'cloudCover' as they are deemed unnecessary.

df = df.drop(['icon', 'summary', 'cloudCover'], axis=1)
Convert to the format: Name, Time, Value
df = df.melt(id_vars=['time'], var_name='name', value_name='value')

Once the data frame is restructured as shown in the image below, it will be ready for upload to Machbase Neo.

3. Data Upload

Finally, the data can be uploaded to Machbase Neo using the command below.

machbase-neo shell import --input ./datahub-2024-1-home.csv.gz --compress gzip --header --method append --timeformat ns home

Check the entire code.

datahub/dataset/2024/01.Smart Home Dataset/conv/convert.py at main · machbase/datahub
All Industrial IoT DataHub with data visualization and AI source - machbase/datahub

4. Check the results after uploading

Output when executing the following code in the Machbase Neo internal shell.

select * from v$home_stat;

※ Subsequent link to the AI training process: Kaggle Smart Home Data

Back to Top