New Tutorials! (#5538)

* added tutorials section to docs

* pushing some initial changes

Rill tutorial courses

- I had to run npm install sidebar to get the tutorial page loaded but it added some dependencies, i think this is fine?

* 100 course backbone

Added some blank pages to start the backbone for 100 courses

* 100 courses

continued building out the 100 courses from Notion,

have some Q's around article length and whether we want to keep the height of all the articles uniform so the 'next' button doesn't vary... thoughts?

* Added up till 103

Continuing adding contents till 103.

Once dataset is ready, will need to go back and add details and media contents.

* finished outline for 100 series

Will need to return to add the details but probably good enough for POC?

* Adding some Gifs instead of Loom

Feedback required: What do you think of the GIFs instead? I do think it looks cleaner and loads faster. We could either host them in our website or in gcs/public-data

* Added GIFs up till "102"

* Added support for UI

Added some blank lists for the UI experience when ready to be added [can make it hidden until needed]

* Pushing last changes for the week

* Actual last push :)

* Some media changes,

* rename to learn and add doc ratings

* fixed tutorials -> learn

* 200 courses pages

some extra stuff for early 200 courses..

* rename to Learn fix

* Always more to push

Always forget to push stuff,

re-created the GIFs for a cleaner UI.

* More gifs, metric-view course updates

* Adding further backbones of 202 series

some UI updates coming in 0.48. so paused on some contents,

outlining some 200 series contents, for chronological storytelling

* Cleaned up all 100 series, 200 remodel

Cleaned up 100s, forgot to add the model's SQL join,

added `coming soon` to the UI related stuff

getting confirmation on adding `fun` GIFs.

* fixing Card List exclusion after renaming

* Guides Examples, Rill/CH, 200 level

Added some topics for Guides,
Added some work to Rill/CH - how similar to Rill basics?
Added some further backbone into 200 series

* Adding a Coming Soon component

masked the UI and RillCH courses with coming soon!

* 200s course outline modification

* Just adding more chaos to the courses

came up with some advanced SQL for 200 course models, i think this will be good enough to wrap up the 200 series once the underlying dataset is complete :)

* Pre push changes before UI Deploy

Pushing current changes before working on the changes for 0.48 - UI Deploy

* Small changes, bigger GIFs for visibility

* 200 series + new GIFs

Further building out 200 series, recreated GIFs at width:800px

* last push for the week

cleaning up a few things.

* starting 301 content

* Almost done with 200 series :)

Bunch of 200 series content finished, with media not included.

Need to go back to make the descriptions and media for the cloud features, but for the most part have everything ready to showcase for FE offsite.

Will soon start to work on the Guides section to write out some simple guides :)

* changed Comingsoon from ID to CLASS,

allows for multiple coming soon message per page,

* Adding 301 in split sections to follow rest of the pattern we've used

* custom dashboard updates

* Finishing up 200 series, pretty much ready for FE offiste

Got some good ideas from Alex for Guide/Blogs

* Added further contents to Rill_CH guide, sidebar actions for Guide and Learn to be clickable

* Finishing up Rill/ClickHouse course outline with contents

Will need to review this and make it a bit better,
I have added snowflake to Clickhouse staging connection **without** incremental modeling, debating if worth discussing the topic here, probably not.

* Adding contents to /guides/

- S3/GCS/ABS/BQ to Rill
same content different links

- Average of Averages

@alextautu im thinking of making some changes to the 300 custom dashboard content, that okay?

* Modified 300 series custom-dashboard

Modified the custom dashboard to use the CH dataset as this allows for me to walk through from start to end a full custom dashboard with charts, etc. THen when uploading the final project, they have a single project for reference :)

* Added a walkthrough for split, incremental models, and staging

* Fixed datasets

Changes to the format of where GCS files are saved, need to change sources in 100 series.

* changed the s3 files to our CH repo files

Since Alex updated the folder path, Im using CH repo data instead of sample data for the course.

* Small last fixes, added video section with CONCEPTs

* blank guides and fixing

* changes to learn -> Tutorial

* change to tutorials, reworking categories

* big changes to everything

reformat

* Should be ready for Review

- remove embedd dashbaord from Rill Advanced (this doesn't exist in the repo, its for external, well documented in the docs)

- Created Administration
- User Management
o creating Users
o Creating User groups
- Project Management
o Project Maintanence
o Credential Managment
o GitHub

* Update sidebars.js

* Update credential-mangement.md

* fixes

* IDK why a few things broke but repushing

* npm run build ran successfully locally

* fixing package.json

* Adding a few more admin training topics

* fixed 100 / 300 series

* some reorganization and changes suggest in Notion

* Added Custom Dashboard note

* added env variable management

Based on PRD for upcoming UI changes, added `coming soon` for a few sectinons and added variable management to admin tutorials

* How to add New Dimension guide

* fix: rename custom dash and prevent collapse of tutorials header

* fix: broken sidebar ref

* minor changes

* Edits to Canvas Dashboard tutorial

Still needs examples for all the new charts

* adding list users to admin course

* adding tags

* Small edits

* Separated Alerts into a new admin tutorial page

Added a failing dashboard and alerts. I dont think alerting is available in Rill Cloud yet so will wait for 0.49 release then add screenshot of email.

* fixed dashboard YAML

* Update alerts.md

* adding alerts to admin page

* Removing canvas dashboard section for now

As discussed in
https://rilldata.slack.com/archives/C07EBKZR2V7/p1726585781765539?thread_ts=1726579146.635389&cid=C07EBKZR2V7

once tutorials are live, will pull changes to canvas dashboard branch and uncomment the section.

* Coming soon for incremental datawarehouse

We dont quite have a data warehouse ready to be used publicly so will make it coming soon!

* excluding the advanced features section

Seems like a lot of changes coming from canvas, API, and incremental.

Instead of having confusing changing information, will release this section when clarified!

* Delete docs/docs/deploy/existing-project/existing-project.md

* Delete add-custom-dashboard.png

* Create add-custom-dashboard.png

* Delete add-custom-dashboard.png

---------

Co-authored-by: Alex Tautu <[email protected]>
Co-authored-by: Alexander Thor <[email protected]>

.gitignore

@@ -79,3 +79,4 @@ rill-binary-sa.json
 sysroot/
 latest.txt
 /web-common/coverage
+package-lock.json

docs/docs/build/olap/olap.md

@@ -39,6 +39,7 @@ Rill will use DuckDB by default as an embedded OLAP engine but it is **not** cur
 
 :::
 
+
 ## Druid
 
 When Druid has been configured as the [default OLAP engine](../../reference/project-files/rill-yaml.md#configuring-the-default-olap-engine) for your project, any existing external tables that Rill can read and query should be shown through the Rill Developer UI. You can then create dashboards using these external Druid tables.

docs/docs/manage/usergroup-management.md

@@ -4,6 +4,8 @@ sidebar_label: User Groups
 sidebar_position: 22
 ---
 
+import ComingSoon from '@site/src/components/ComingSoon';
+
 ## Managing User groups 
 
 In Rill Cloud, access to projects can be granted via user groups via the CLI. 

docs/docs/reference/olap-engines/clickhouse.md

@@ -18,6 +18,12 @@ Rill supports connecting to an existing ClickHouse instance and using it as an O
 
 Rill supports connecting to ClickHouse v22.7 or newer versions.
 
+## ClickHouse Local vs ClickHouse Cloud
+
+:::
+UPDATE THIS
+:::
+
 ## Connection string (DSN)
 
 Rill is able to connect to ClickHouse using the [ClickHouse Go Driver](https://clickhouse.com/docs/en/integrations/go). An appropriate connection string (DSN) will need to be set through the `connector.clickhouse.dsn` property in Rill.

docs/docs/tutorials/_category_.yml

@@ -0,0 +1,5 @@
+position: 200
+label: " "
+collapsible: true
+collapsed: true
+className: hidden

docs/docs/tutorials/administration/alerts.md

@@ -0,0 +1,71 @@
+---
+title: "Alerting"
+description: Project Maintanence
+sidebar_label: "Alerting"
+tags:
+ - CLI
+ - Administration
+---
+
+There are two different types of alerting in Rill. 
+
+1. [Project Error Alerting](https://docs.rilldata.com/deploy/project-errors)
+2. [Dashboard Alerting](https://docs.rilldata.com/explore/alerts/)
+
+## Project Error Alerting
+
+When deploying to Rill Cloud, there might be a few different reasons why a project goes into an error state. However, this may not be obvious during deployment. Therefore, there is an option to create project alerting based on some sort of query that you can derive from your project. 
+
+
+### Setting up the YAML file in Rill Developer
+
+Let's go ahead and create a basic alert on the project that sends an email if any of the resources reconcile with error.
+[]link the alert YAML once out.
+The default alerting is:
+```yaml
+type: alert
+
+# Check the alert every 10 minutes.
+refresh:
+ cron: "*/10 * * * *"
+
+# Query for all resources with a reconcile error.
+# The alert will trigger when the query result is not empty.
+data:
+ resource_status:
+ where_error: true
+
+# Send notifications by email
+notify:
+ email:
+ recipients: [[email protected]]
+```
+
+So as not to spam our own inbox, let's change the alert to run the 1st of every month by changing the CRON to:
+```
+ cron: "0 0 1 * *"
+```
+
+Once complete, we can go ahead and create a broken dashboard or model. The easiest would be to create a new dashboard via +Add, and leaving it as is. There will be a solid red border around the text editor. 
+![img](/img/tutorials/admin/new-dashboard.png)
+
+
+Once we've created this, let's [push our changes to Rill Cloud](/tutorials/rill_advanced_features/advanced_developer/update-rill-cloud). 
+
+
+![img](/img/tutorials/admin/failing-dashboard.png)
+
+Now, you'll receive an email that gives you more information on the failing resource, in this case the dashboard. 
+
+
+![img](/img/tutorials/admin/alert-email.png)
+
+You can view all the alerts (project and dashboard) from the Alerts page
+
+![img](/img/tutorials/admin/alert-code.png)
+
+## Dashboard Alerts
+
+[Alerts created on individual dashboards](https://docs.rilldata.com/explore/alerts/) can be viewed from a project's alert page. As an admin, you can edit or delete the alert as needed.
+
+![alerts](/img/tutorials/admin/alert-admin.png)

docs/docs/tutorials/administration/credential-envvariable-mangement.md

@@ -0,0 +1,133 @@
+---
+title: "Credential and Environmental Variable Management"
+description: how to use credentials in Developer vs Cloud
+sidebar_label: "Credential and Environmetal Variable Management"
+tags:
+ - CLI
+ - Administration
+---
+import ComingSoon from '@site/src/components/ComingSoon';
+
+
+Please review the documentation on [Credential Managment](https://docs.rilldata.com/build/credentials/) and [environmental variable](https://docs.rilldata.com/build/credentials/#variables) / [templating](https://docs.rilldata.com/deploy/templating) before getting started.
+
+
+## Managing Credentials and Variables on Rill Cloud 
+<ComingSoon />
+
+<div class='contents_to_overlay'>
+Historically (pre 0.48), management was only possible via the CLI. Now, it is also possible to do so via the UI! 
+
+</div>
+
+## Credentials
+
+There are various ways that you can set credentials in Rill. While each has its pros and cons, there are some key differences between the setup methods. 
+
+1. via the CLI, utilizing `rill env` 
+2. via an automatically created `.env` file when running `rill env pull`
+3. Manually defining variables in rill.yaml or the source.yaml
+4. Manually defining variables when running `rill start`
+
+For 1, 2, and 3, these methods will allow Rill Cloud to access the sources. However, method 3 will have your credential details in plaintext in GitHub (assuming you have synced to a GitHub Repository). Therefore we would recommend using method 1 and 2 when deploying your project to Rill Cloud. For local Development purposes, you can either pull the credentials from Rill Cloud (assuming its already been deployed) or by passing the key and value.
+```rill start --var key=value```
+
+
+
+### Configuring Credentials via `rill env configure`
+
+Assuming that you have worked through all of the courses up until now, when running the following command you will get prompted for the following:
+
+```bash
+rill env configure
+Finish deploying your project by providing access to the connectors. Rill requires credentials for the following connectors:
+
+ - bigquery (used by SQL-incremental-tutorial)
+ - clickhouse (used by staging_to_CH)
+ - gcs (used by commits__ and others)
+ - s3 (used by staging_to_CH)
+ - snowflake (used by staging_to_CH)
+```
+
+When running locally, Rill will pull environmental credentials or use locally defined credentials. However, when pushing to Rill Cloud, not all of these are available so you will need to define these. 
+
+### Authenticating
+Please refer to each connection's documentation on what is required to authenticate. 
+
+```bash
+Configuring connector "bigquery":
+For instructions on how to configure, see: https://docs.rilldata.com/reference/connectors/bigquery
+? connector.bigquery.google_application_credentials (Enter path of file to load from.) 
+
+Configuring connector "clickhouse":
+For instructions on how to configure, see: https://docs.rilldata.com/reference/olap-engines/clickhouse
+? connector.clickhouse.host 
+? connector.clickhouse.port 
+? connector.clickhouse.username 
+? connector.clickhouse.password 
+? connector.clickhouse.ssl 
+
+Configuring connector "gcs":
+For instructions on how to configure, see: https://docs.rilldata.com/reference/connectors/gcs
+? connector.gcs.google_application_credentials (Enter path of file to load from.) 
+
+Configuring connector "s3":
+For instructions on how to configure, see: https://docs.rilldata.com/reference/connectors/s3
+? connector.s3.aws_access_key_id 
+? connector.s3.aws_secret_access_key 
+
+Configuring connector "snowflake":
+? connector.snowflake.dsn 
+```
+
+Once you have completed, this you can head back over to Rill Cloud and check if your sources are connecting successfully. 
+
+
+### Making changes to the `.env` file.
+
+This file is automatically generated when pulling your credentials from a Rill Cloud deployed project.
+
+```bash
+rill env pull
+Added .env to .gitignore.
+Updated .env file with cloud credentials from project "my-rill-tutorial".
+```
+
+When starting Rill Developer, you will see a new file .env that has all the defined credential in text. You can make any required changes to this file and update the credentials in Rill Cloud by running:
+
+```bash
+rill env push
+```
+
+This file is not saved in plain text in your GitHub Repository.
+
+
+## Environmental Variables
+Similar to credentials, Environmental variables are usually set on Rill Developer and pushed to Rill Cloud when ready for deployment.
+
+### Configuring Environmental Variables
+There are a few ways to use environmental variables locally.
+
+1. [Defining key-value pairs](https://docs.rilldata.com/reference/project-files/rill-yaml#setting-variables) in the `rill.yaml`.
+2. If already deployed to Rill Cloud, directly modifying the .env file or using `rill env set`.
+3. via the rill start command
+```bash
+rill start --var <var_name>=<value>
+```
+
+### Pushing to Rill Cloud
+
+Depending on the method used during development, pushing to Rill Cloud can be as easy as:
+
+1. Updating the Rill Deployment (if you added the variables to `rill.yaml`)
+2. Pushing the .env file after making changes by running `rill env push`.
+3. Setting the rill env variables by running `rill env set <key> <value>`
+
+> Each number corresponds to how you configured the environmental variable above.
+
+
+
+import DocsRating from '@site/src/components/DocsRating';
+
+---
+<DocsRating />

docs/docs/tutorials/administration/github.md

@@ -0,0 +1,119 @@
+---
+title: "GitHub"
+description: Connect to GitHub [note on status page]
+sidebar_label: "GitHub"
+---
+
+
+## Connect to GitHub
+After deploying to Rill Cloud, you will have the ability to sync your project to a GitHub repository. 
+
+### via UI
+
+<img src = '/img/tutorials/203/github-ui.gif' class='rounded-gif' />
+<br />
+
+Starting from `v0.48`, we have added a new feature to deploy your project directly from the UI. By selecting the `Connect to GitHub` button you will then need to follow the steps in the UI to create a new blank repository [or a select an existing one] and connect this to your current Rill project. Rill will then push the current contents [or overwrite]. You'll be direct back to the Status page and can see that your project is now <a href ='https://docs.rilldata.com/deploy/existing-project/' target="BLANK" >synced with your GitHub repository! </a>
+
+Note that you will not get all the same alerts and warning that you do via the CLI, such as source credentials check.
+
+:::tip
+While not shown in the GIF above, you may want to take a moment to set up the[ GitHub repository connection to your local folder](https://docs.rilldata.com/deploy/existing-project/github-101).
+:::
+---
+### via CLI
+
+```bash
+rill project connect-github --help
+Deploy project to Rill Cloud by pulling project files from a git repository
+
+Usage:
+ rill project connect-github [flags]
+
+Flags:
+ --path string Path to project repository (default: current directory) (default ".")
+ --subpath string Relative path to project in the repository (for monorepos)
+ --remote string Remote name (default: first Git remote)
+ --org string Org to deploy project in (default "Rill_Learn")
+ --name string Project name (default: Git repo name)
+ --description string Project description
+ --public Make dashboards publicly accessible
+ --provisioner string Project provisioner
+ --prod-version string Rill version (default: the latest release version) (default "latest")
+ --prod-branch string Git branch to deploy from (default: the default Git branch)
+```
+
+Navigating back to Terminal, we can run the following:
+```
+rill project connect-github
+No git remote was found.
+? Do you want to create a repo? Yes
+? Select a Github account for the new repository royendo
+
+Request submitted for creating repository. Checking completion status
+
+Successfully created repository on "https:/royendo/my-rill-tutorial-cli"
+
+Pushing local project to Github
+
+Successfully pushed your local project to Github
+
+Using org "Rill_Learn".
+
+Rill project names are derived from your Github repository name.
+Created project "Rill_Learn/my-rill-tutorial-cli". Use `rill project rename` to change name if required.
+
+Rill projects deploy continuously when you push changes to Github.
+Your project can be accessed at: https://ui.rilldata.com/Rill_Learn/my-rill-tutorial-cli
+Opening project in browser...
+```
+
+The CLI will ask if you want to create a repo, select Yes. 
+
+If this is your first time, you will be prompted to log into Github. Once completed, you'll see this in the browser and can navigate back to the CLI.
+
+![img](/img/tutorials/203/git_okay.png)
+
+Rill will automatically use the project name as the repository and project name. If there are any issues with overlapping names, it will prompt your for a different name. Once this all completes, your browser will be automatically opened.
+
+
+![img](/img/deploy/existing-project/cli-upload.png)
+
+That's it! You have connected your GitHub repository to your Rill project. Now navigating back to the Status page, you can see the repository listed. Now you can push any changes that you've made locally to the Git repository and Rill will automatically update. For more information on how to use GitHub with Rill, please refer to the <a href= 'http://localhost:4004/deploy/existing-project/github-101#pushing-changes' target ="blank" > GitHub Basics docs</a>!
+
+
+## Making Changes to the GitHub Repository
+While this is an unusual step and most changes and development to your project should be done via branches on your Repository, there might be times you may need to completely change the repository.
+
+### Via the UI
+
+By selecting the edit button (blue pencil) next to the GitHub Repository, the following UI will open where you can change the repository. Note that this will **push** the contents of your Rill project to the repository. You can change the branch of the repository under Advanced options.
+
+![img](/img/tutorials/admin/edit-github.png)
+
+If you want to **pull** the contents of your new repository to Rill Cloud, you will need to do the following:
+
+1. Pull contents of GitHub Repository locally.
+2. (Optional) If you want to keep the same project name, you will need to change the current project name by running `rill project rename --project <project_name>`.
+ - WARNING: Renaming a project will invalidate dashboard URLs, including public URLs
+3. Deploy the new folder by running `rill start` and select `Deploy` from the dashboard
+4. After confirming that the new project works, delete old project by running `rill project delete <project_name>`
+
+
+### Via the CLI
+
+You can redeploy a project by re-running the `rill project connect-github` with the new repository. Note that if you want to keep the same name of the project, you will need to rename the project before doing so.
+
+**Modifying the Branch**
+```bash
+rill project edit 
+? Select project my-rill-tutorial
+? Enter the description 
+? Enter the production branch main
+? Make project public No
+```
+
+import DocsRating from '@site/src/components/DocsRating';
+
+---
+<DocsRating />