Add dashboard for timeseries management

[paulapreuss]

Will (hopefully) close #403, close #401, close #331, close #332.

The current state is simply a frontend prototype to display what this dashboard could look like (a lot of this code will have to be deleted and replaced). Here are some of my thoughts on the things that need to be handled (alternative approaches of course are also okay):

Backend:

• Currently, every time a Timeseries is used for a different scenario, a duplicate timeseries gets created, that means that a timeseries might have four duplicates (for each scenario) with the exact same values. It might be better to replace the scenario field of the Timeseries with project and fill those with scenario.project on the next migration (keeping only one Timeseries), which would get rid of the many duplicates. This would need a custom migration to handle the deletion and replacement of the field values - here we have to be really careful since we would be deleting production database objects. We still might have the issue of a user wanting to share a timeseries between different projects. The options here would be to handle it through a ManyToMany field instead of a strict ForeignKey (a Timeseries can be assigned to multiple Projects) or to simply have the user duplicate this timeseries for the other project (I think this is my preffered option, as this would also avoid issues if for example one project has a different number of timesteps than the other).
• The Timeseries class also should get two new metadata fields: a generation_parameters JSONField (stores the values used to create the timeseries, e.g. if a timeseries is created from an eesyplan method with a specific tilt/azimuth it can store these inputs in a JSON format), and a description TextField where the user can freely add a description to their Timeseries. It was wished that this description would also be shown in the modal when selecting a timeseries on an asset.

Frontend:

• The timeseries dashboard should easily reachable. I'd say for now we can put it on the top navigation bar, next to "My Projects".
• The current implementation is all dummy for display with the project partners, so it all needs to be hooked up to the backend
  • For the edit Timeseries as well as for the upload Timeseries buttons, I think we can create a TimeseriesModelForm out of the Timeseries class and feed it to the existing modal template ({% include "modal_template.html" ... %})
  • The modal will need the option to upload a file like the current timeseries_input does, and should make use of the validation pipeline that is already established there
  • The delete button can be fairly straightforward (doesn't need a modal) but should include a very explicit warning message along the lines of "Are you sure? This will delete the selected timeseries from all scenarios it is currently used in. You may have to reconfigure the respective assets."
  • The "Create Manually" will be hooked up later when it is finalized which eesyplan methods should be accessible for creating Timeseries and which inputs they require.
• There needs to be options to filter the table (by project, by asset type, by length etc.)
• Later we need to change some things (e.g. the selection of the timeseries should be dynamic) and things should look a bit nicer. I think we might just have Bryan look at it once it is somewhat ready though, so no need to spend too much time on this now.

[josihoppe]

When testing the validation, I saw that the validation, if the length of the timeseries matches the length of the timesteps in the scenario/projects, is not working yet. That is due to the fact, that atm you cannot connect your timeseries to a scenario yet.

Since the whole timeseries to scenario relation will be changed to a timeseries to project relation, the fixing of the length mismatch validation will wait until that is changed.

Also there is the question of, if you wanna have timeseries that will not belong to any project or if every timeseries will need to have a project. Having timeseries without a project gives the advantage, that you should be able to use them in any project/scenario. In that case the length mismatch check will need to be adjusted.

[paulapreuss]

@CelinaKellinghaus @josihoppe looking good so far. I just had a look at the actual dash and have a few additional notes that came up:

• Scenario (in the future Project) definitely also needs a column on the timeseries table. However, we can remove the "steps" column, as only hourly timesteps are currently allowed anyway, I'm pretty sure. This can just be done in parallel with the filtering logic.
• For the timeseries upload modal: We can also hide the timestep field here for now, same with the unit (I don't think we have implemented changing the timeseries based on the unit), so the user has to upload in kW for now. The category field can also be hidden, as we can infer it from the asset type, which gives more information. We can add some small logic to the clean method of this form that infers the category based on the asset type.
• The form fields need formatting with the help text tooltips, explanation texts and translation support like the other fields do. To check how this works currently, check out OpenPlanModelForm (which the other forms inherit from) and the information defined in mvs_parameters_list.csv (the name is a bit obsolete currently but the functionality is what matters).

[CelinaKellinghaus]

@paulapreuss
I couldn't find an existing mapping from asset_type to category. I wrote a simple dict of my own for now, but I'm not confident it matches your intended logic. Happy to adjust once we've discussed it.

Similarly, for the tooltip/help text task: I switched TimeseriesModelForm to inherit from OpenPlanModelForm and drafted new rows for MVS_parameters_list.csv. I've made these changes locally but am holding off on pushing until we've talked, since I'm not sure that I understood this task as intended.

[paulapreuss]

I cherry-picked these changes for #511, which is also still WIP. We will have to check where to keep the migrations depending on the merge order (just something to keep in mind).

[paulapreuss]

Here, you are redirecting the user back to timeseries_dashboard regardless of if the form is valid or not. If the form is not valid, the modal should not close but instead display the validation error message so that the user knows that something went wrong, like it does in the asset modals when building the scenario. This is a bit tricky because you have to dynamically update the form contents to display the error instead of reloading the page. You can check how it is done for the asset modals in grid_model_topology.js submitForm (starting from line 374)

[paulapreuss]

Same comment for timeseries_upload, needs to take data validation and feedback into account.

[josihoppe]

I am still working on that redirecting and displaying the form errors, but should be working tomorrow

[paulapreuss]

Good start, only that the validation error is currently not being passed to the user. Right now I'm able to upload any random file and simply be redirected back to the dashboard.

[paulapreuss]

I am having the issue that the filtering options are repeating instead of showing up once per option

I assume that the .distinct() option here is meant to avoid this, but something is not working quite right yet

[paulapreuss]

I think the filter looks pretty nice, however I think that it still needs a bit of fine-tuning in terms of usability. Some things I noticed:

• It should be possible to filter by multiple options, e.g. to filter all PV timeseries from a specific project. Not sure what the nicest way to do this would be UI wise - in general I do like the current filtering box
• When clicking on a timeseries, the page resets and displays all timeseries again, also resetting the filtering options that we just set. This is an issue based on the fact that we currently reload the whole page when selecting any timeseries, when actually here we should be working with javascript to dynamically update the table contents and the timeseries details without refreshing (we probably need at least two functions to fetch and replace innerHTML of the table/details - the corresponding views would only return the JSON data to javascript instead of redirecting)

[paulapreuss]

I think this is good, couldn't find any other place where this is already defined either :)

[paulapreuss]

I think for the scenario filter the option to select should display the scenario name (currently it displays the scenario number which is not that helpful). When applying the option to the filter, though, it probably makes more sense to pass the ID, so I would probably pass existing scenarios as tuples (id, name) to allow the user to select by name but internally use the ID for the filtering.