Skip to content

WIP: feature: Add fetch-spacetrack-hist#2

Open
kerel-fs wants to merge 1 commit intogalactics:masterfrom
kerel-fs:pr/add_spacetrack_hist
Open

WIP: feature: Add fetch-spacetrack-hist#2
kerel-fs wants to merge 1 commit intogalactics:masterfrom
kerel-fs:pr/add_spacetrack_hist

Conversation

@kerel-fs
Copy link

@kerel-fs kerel-fs commented Dec 12, 2019

It's working but there are several todos:

  • evaluate if the new dependency on spacetrack is required. If yes, consider re-writing the existing spacetrack code to use this dependency as well
  • remove the tle_dir-hack, and split-up space/tle.py instead (it's >0.5k lines already)
    • extract TleDb
    • separate routing / command parsing from command implementation

Issues:

  • sometimes: peewee.IntegrityError: UNIQUE constraint failed: tlemodel.norad_id, tlemodel.epoch
  • when downloading large datasets: peewee.OperationalError: too many SQL variables

@galactics galactics self-assigned this Dec 29, 2019
@galactics galactics self-requested a review December 29, 2019 13:26
@galactics
Copy link
Owner

Wow ! My first pull request !

This feature can certainly be useful, and you raise a good point on the size of the tle.py file.
I totally missed the existence of the spacetrack package, it should certainly ease some work.

Concerning the issues, I had the same ones, but as I rarely download large files from space-track, this did not bother me that much. I will look more closely into this, but I'm afraid that the only way to manage this would be to split the files in batches on few 1000's TLEs. That or change the whole database structure, which was not designed to handle that much data insertion, as my primary source is celestrak.

Anyway, thank you very much for this proposal. I will try to review it during the next few days. I ask for your patience, as I'm not very used to the github process for handling PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants