Skip to content

Latest commit

 

History

History

Use Case A Graphical exploration

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
Figs
Figs
 
 
Power_BI
Power_BI
 
 
R Shiny files
R Shiny files
 
 
README.md
README.md
 
 

README.md

Use Case A - Graphical exploration.

  • The first code is an R Shiny application allowing the navigation in themes, sub-themes and categories and the display of corresponding articles. There are also some filters for the display of articles, for the year of last update and for keywords in titles and abstracts. Each column in the results (title, URL, year, abstract) has a filter allowing further drill-down, with highlighting of the matched terms.
  • The source code (app.R) is in folder Shiny files. A short documentation is given here.
  • The same folder contains the Python code for the connection with the database and the production of the input file (Use Case A prepare data_v2_rev_Jan2022.ipynb), together with a recently produced file (SE_df_7_15_13_21.xlsx).
  • To run the code producing the input file, please replace user_name and password with your credentials in command:
    c = pyodbc.connect('DSN=Virtuoso All;DBA=ESTAT;UID=user_name;PWD=password'). See also installation instructions in this notebook.
  • The current functionalities can be seen in the online version here. Because of frequent changes, please press Reload in your browser if you have visited this link already.

  • The second code is an MS Power BI report allowing again the navigation in themes, sub-themes and categories and the display of corresponding articles. There is also an interactive wordcloud graphical element and a network element connecting themes with sub-themes. Another page displays the database structure in a treemap, together with the dataset names and download links for the selected section. A fourth page links the topic modeling results with themes, sub-themes, categories and articles.
  • A short documentation is given here. The following data requirements are explained in the documentation but also mentioned here:
  • The current .pbix file is in folder Power BI. The same folder contains:
    • The Python code for the connection with the database and the production of the input files (Use Case A prepare data_v2_rev_Jan2022.ipynb), together with two recently produced files (SE_df_2_1_12_43.xlsx and Crumbs_2_1_12_44.xlsx).
    • This code also produces, in Excel files, two direct exports from the topic modelling results with the SE articles currently in the content database:
      • Table ESTAT.V1.tm_topics into tm_topics.xlsx - a look-up table for the topics, their keywords and their descriptions.
      • Table ESTAT.V1.tm_articles_to_topics into tm_articles_to_topics.xlsx - this contains the associated distributions of articles into topics).
  • To run the Python code producing the input file from the database (Use Case A prepare data_v2_rev_Jan2022.ipynb), please replace user_name and password with your credentials in command:
    c = pyodbc.connect('DSN=Virtuoso All;DBA=ESTAT;UID=user_name;PWD=password'). See also installation instructions in this notebook.
  • The current functionalities can be seen in the online version here. Because of frequent changes, please press Reload in your browser if you have visited this link already.