Leonard Excoffier
|
e17a54ba3c
|
feat: advancing on new script
|
2024-09-10 22:23:14 -04:00 |
|
Leonard Excoffier
|
e586c2b202
|
feat: downloads and extract the zip.
|
2024-09-10 21:38:43 -04:00 |
|
Leonard Excoffier
|
5e38f46e8f
|
Temp expirimentation, problem with duplicate primary key lowercase uppercase and insert on duplicate, need to find way to bulk insert but still not insert rows that have duplicate.
|
2024-08-31 22:34:04 -04:00 |
|
Leonard Excoffier
|
162106c8e0
|
e
|
2024-08-31 19:50:15 -04:00 |
|
Leonard Excoffier
|
fa64f81cb8
|
clean slate
|
2024-08-31 19:14:02 -04:00 |
|
Leonard Excoffier
|
91272506e3
|
feat: drops columns that are not supposed to be there
|
2024-08-31 18:54:28 -04:00 |
|
Leonard Excoffier
|
3f5f6d5569
|
fix: remove not null in some places
|
2024-08-31 18:26:54 -04:00 |
|
Leonard Excoffier
|
84b20e1496
|
fix: removed NOT NULL specification for non primary keys
|
2024-08-31 18:26:14 -04:00 |
|
Leonard Excoffier
|
4c9b2e46c1
|
fix: put the doc type back to TEXT
|
2024-08-31 18:23:48 -04:00 |
|
Leonard Excoffier
|
c9612b35ae
|
fix: added some special fixe that are not taken into account by dataset documentation
|
2024-08-31 17:18:33 -04:00 |
|
Leonard Excoffier
|
025ba9b913
|
fix: fixed coreg being a null value issue
|
2024-08-31 17:17:17 -04:00 |
|
Leonard Excoffier
|
57b13dbec5
|
feat: based on the sec doc match the sql schema
|
2024-08-31 16:02:39 -04:00 |
|
Leonard Excoffier
|
00a59b27ce
|
feat: now writes all datasets to db
|
2024-08-31 15:52:37 -04:00 |
|
Leonard Excoffier
|
015879ef4d
|
feat: added reset block
|
2024-08-31 15:45:53 -04:00 |
|
Leonard Excoffier
|
3a60674b7c
|
feaet: now cycles through wanted files
|
2024-08-31 15:41:33 -04:00 |
|
Leonard Excoffier
|
f31d901201
|
refactor: total change, no longer ripping gigabytes but going directly to datasets, must find way to automate download.
|
2024-08-31 15:32:57 -04:00 |
|
Leonard Excoffier
|
b5e0af3ca5
|
feat: added some helpfull commands
|
2024-08-31 13:00:04 -04:00 |
|
Leonard Excoffier
|
ab8f1fcf8f
|
feat: now use batches but seem to write less rows, not sure
|
2024-08-31 00:50:17 -04:00 |
|
Leonard Excoffier
|
9c5afa7670
|
feat: script cycle works
|
2024-08-31 00:38:50 -04:00 |
|
Leonard Excoffier
|
765272f896
|
feat: seem to be able to write to a db.
|
2024-08-31 00:17:37 -04:00 |
|
Leonard Excoffier
|
0fbbb37ef7
|
feat: design new schema
|
2024-08-30 19:59:42 -04:00 |
|
Leonard Excoffier
|
71a054e2b6
|
feat: new script seem to be super efficient
|
2024-08-30 19:54:37 -04:00 |
|
Leonard Excoffier
|
bfa36303af
|
feat: ignored defautl sql file
|
2024-08-30 18:35:14 -04:00 |
|
Leonard Excoffier
|
6fcbc73a4f
|
fix: correctly adapted script for new primary key
|
2024-08-30 08:16:05 -04:00 |
|
Leonard Excoffier
|
184f82a688
|
fix: correctly set primary key for data table
|
2024-08-30 07:56:02 -04:00 |
|
Leonard Excoffier
|
0f9a42eb7e
|
feat: now checks if cik is already present before writing to db, can be deactivated if want to update data.
|
2024-08-29 22:04:47 -04:00 |
|
Leonard Excoffier
|
a17d73f336
|
feat: now loops over all the facts jsons
|
2024-08-29 22:03:05 -04:00 |
|
Leonard Excoffier
|
5946ff73bd
|
feat: script to write info to db seems to work, have no idea how tho, need to generalize with a for loop for all files
|
2024-08-29 21:51:05 -04:00 |
|
Leonard Excoffier
|
750d33b976
|
feat: added stuff and corrected the schema creation.
|
2024-08-29 21:39:57 -04:00 |
|
Leonard Excoffier
|
d5d823b290
|
feat: changed primary key of facts table
|
2024-08-29 21:35:30 -04:00 |
|
Leonard Excoffier
|
97bd73c4a3
|
fix: stopped download and extraction of submissions, only keep company facts
|
2024-08-27 22:11:27 -04:00 |
|
Leonard Excoffier
|
4266e09691
|
feat: script to output all the labels , initiation to parsing jsons, may use jquery on mac.
|
2024-08-27 21:58:16 -04:00 |
|
Leonard Excoffier
|
ce0152bbba
|
feat: completed db schema
|
2024-08-27 19:13:07 -04:00 |
|
Leonard Excoffier
|
19a5579fb0
|
feat: db schema has now a base
|
2024-08-27 19:09:12 -04:00 |
|
Leonard Excoffier
|
c02fea1417
|
feat: initiated description of db schema based on facts json.
|
2024-08-26 22:08:28 -04:00 |
|
Leonard Excoffier
|
20a2866ecd
|
batman
|
2024-08-25 19:52:34 -04:00 |
|