15 Commits

Author SHA1 Message Date
Leonard Excoffier
5e38f46e8f Temp expirimentation, problem with duplicate primary key lowercase uppercase and insert on duplicate, need to find way to bulk insert but still not insert rows that have duplicate. 2024-08-31 22:34:04 -04:00
Leonard Excoffier
162106c8e0 e 2024-08-31 19:50:15 -04:00
Leonard Excoffier
fa64f81cb8 clean slate 2024-08-31 19:14:02 -04:00
Leonard Excoffier
91272506e3 feat: drops columns that are not supposed to be there 2024-08-31 18:54:28 -04:00
Leonard Excoffier
84b20e1496 fix: removed NOT NULL specification for non primary keys 2024-08-31 18:26:14 -04:00
Leonard Excoffier
025ba9b913 fix: fixed coreg being a null value issue 2024-08-31 17:17:17 -04:00
Leonard Excoffier
00a59b27ce feat: now writes all datasets to db 2024-08-31 15:52:37 -04:00
Leonard Excoffier
3a60674b7c feaet: now cycles through wanted files 2024-08-31 15:41:33 -04:00
Leonard Excoffier
f31d901201 refactor: total change, no longer ripping gigabytes but going directly to datasets, must find way to automate download. 2024-08-31 15:32:57 -04:00
Leonard Excoffier
6fcbc73a4f fix: correctly adapted script for new primary key 2024-08-30 08:16:05 -04:00
Leonard Excoffier
0f9a42eb7e feat: now checks if cik is already present before writing to db, can be deactivated if want to update data. 2024-08-29 22:04:47 -04:00
Leonard Excoffier
a17d73f336 feat: now loops over all the facts jsons 2024-08-29 22:03:05 -04:00
Leonard Excoffier
5946ff73bd feat: script to write info to db seems to work, have no idea how tho, need to generalize with a for loop for all files 2024-08-29 21:51:05 -04:00
Leonard Excoffier
750d33b976 feat: added stuff and corrected the schema creation. 2024-08-29 21:39:57 -04:00
Leonard Excoffier
4266e09691 feat: script to output all the labels , initiation to parsing jsons, may use jquery on mac. 2024-08-27 21:58:16 -04:00