Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor docs; minimal content change #621

Merged
merged 4 commits into from
Jan 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/concepts/customizing-execution.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Customizing Execution
#####################

The following tools are all part of the lifecycle customization capability in Hamilton. This is a very powerful set of features,
a few of which are presented below. See the reference for :doc:`../reference/customizing-execution/index` for more information.
a few of which are presented below. See the reference for :doc:`../reference/graph-adapters/index` for more information.

You can mix/match lifecycle capabilities, although some may not be compatible. To do so, pass a variable number into the ``with_adapters(...)`` function
of the ``driver.Builder``.
Expand Down Expand Up @@ -72,7 +72,7 @@ that we already have the right abstraction and have not yet exposed it via docum

If you want to tell Hamilton to return something else, we suggest starting with the ``lifecycle.ResultBuilder``
and writing a simple class & function that implements the ``lifecycle.ResultBuilder`` interface and passing that into the driver as ``adapter=[result_builder]``. See
:doc:`../reference/customizing-execution/index` and
:doc:`../reference/graph-adapters/index` and
:doc:`../reference/result-builders/index` for options.

Execution Hooks
Expand Down
2 changes: 1 addition & 1 deletion docs/concepts/driver-capabilities.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ Let's go through some examples that show you how to write a Hamilton function th
depending on configuration.

**You have a DAG for region and business line**, where the rolling average for marketing spend is computed differently
(see :doc:`../getting-started/index` for the motivating example). In this case, you'll define the DAG as follows:
(see :doc:`../get-started/index` for the motivating example). In this case, you'll define the DAG as follows:

.. code-block:: python

Expand Down
6 changes: 0 additions & 6 deletions docs/concepts/extensions.rst

This file was deleted.

3 changes: 1 addition & 2 deletions docs/concepts/index.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
====================
Overview of Concepts
Concepts
====================

Now that you're familiar with the basics and have run your own dataflow, let's dive into the
Expand All @@ -14,4 +14,3 @@ concepts that makes Hamilton unique and powerful.
customizing-execution
decorators-overview
best-practices/index
extensions
154 changes: 0 additions & 154 deletions docs/extensions.rst

This file was deleted.

11 changes: 0 additions & 11 deletions docs/further-reading.rst

This file was deleted.

File renamed without changes.
15 changes: 8 additions & 7 deletions docs/getting-started/index.rst → docs/get-started/index.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
====================
Getting Started
Get Started
====================

Welcome to Hamilton's documentation!
Expand All @@ -18,14 +18,15 @@ It allows you to:
its basic capabilities.


Getting started with Hamilton locally
Get started with Hamilton locally
-------------------------------------
The following section of the docs will teach you how to install Hamilton and get started with your own project.

.. toctree::

install
write-first-dataflow
run-first-dataflow
hamilton-hub
next-steps
why-hamilton
install
your-first-dataflow
learning-resources
contributing
license
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@

Installing hamilton is easy!

# Installing
# Install

Hamilton is a lightweight framework with a variety of extensions/plugins. To get started, you'll need the following:

- ``python >= 3.7``
- ``python >= 3.8``
- ``pip``

For help with python/pip/managing virtual environments see the [python docs](https://docs.python.org/3/tutorial/venv.html/).
Expand Down
111 changes: 111 additions & 0 deletions docs/get-started/learning-resources.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# Learning Resources

Several channels are available to get started with Hamilton, learn advanced usage, and participate in the latest feature development.

## 📒 User Guide Documentation

The [user guide](../concepts/index.rst) gives a complete overview of Hamilton's features.

## 📚 Reference Documentation

The [reference documentation](../reference/dataflows/index.rst) details Hamilton's public API.

## ✍ tryhamilton.dev

The [tryhamilton.dev](https://tryhamilton.dev) website provides interactive tutorials in-browser to learn specific Hamilton concepts.

## 🛠 Dataflow Hub

The [Hamilton Dataflow Hub](https://hub.dagworks.io/docs/) hosts user-created dataflows that are easy to download and reuse in your project.

## 💡 Blog

The [DAGWorks Blog](https://blog.dagworks.io/) publishes articles on problems Hamilton helps solve, reference architectures, and new features.

## 👋 Slack
The [Slack channel](https://join.slack.com/t/hamilton-opensource/shared_invite/zt-1bjs72asx-wcUTgH7q7QX1igiQ5bbdcg) is the ideal place to ask questions, request features, and give feedback.

## 📣 Talks
* 2023-12    Why you should build your GenAI/LLM apps using Hamilton. AICamp End of Year in SF
* [Recording](https://www.aicamp.ai/event/eventdetails/W2023121217)
* [Slides](https:/skrawcz/talks/files/13666470/Why.you.should.build.your.GenAI_LLM.apps.using.Hamilton.pdf)

* 2023-12    Bridging Classic ML Pipelines with the World of LLMs. PyData Global
* [Recording](https://global2023.pydata.org/cfp/talk/3REDA9/)
* [Slides](https:/skrawcz/talks/files/13666479/Bridging.Classic.ML.Pipelines.with.the.World.of.LLMs.1.pdf)

* 2023-09    Hamilton: Natively bringing software engineering best practices to python data transformations. Bay Area Python Interest Group (BAYPIGgies)
* [Recording](https://www.meetup.com/baypiggies/events/296283989/)
* [Slides](https:/skrawcz/talks/files/12785978/BayPIGgies_.Hamilton.Talk.pdf)


* 2023-08    dbt + Hamilton: Enabling you to maintain complex Python within dbt models. MDSFest'23
* [Recording](https://www.mdsfest.com/)
* [Slides](https:/skrawcz/talks/files/12431755/dbt.%2B.Hamilton_.Enabling.you.to.maintain.complex.python.within.dbt.models.pdf)

* 2023-06    Hamilton: an OS tool to add to your LLM App toolbelt. LLM Avalanche.
* [Slides](https:/skrawcz/talks/files/11899349/Hamilton_.an.OS.tool.to.add.to.your.LLM.App.toolbelt.pdf)

* 2023-06    Feature Engineering with Hamilton: Portability & Lineage. Budapest ML Forum June 2023
* [Recording](https://budapestml.hu/2023/en/)
* [Slides](https:/skrawcz/talks/files/11690901/Stefan_Krawczyk_BudapestTalkJune2023_FeatureEngineeringwith.Hamilton_Portability.Lineage.pdf)

* 2023-06    British Cycling Data Platform in Python. Manchester PyData Meetup
* [Slides](https:/skrawcz/talks/files/11899331/PyData.British.Cycling.7.June.2023.pdf)
* Co-presented with Peter Robinson, and Murray Tait.

* 2023-04    Lightweight Lineage with Hamilton. PyData Seattle
* [Slides](https:/skrawcz/talks/files/11399972/PyData-Seattl-Lightning-Talk-2023-Lighweight-Lineage-with-Hamilton.pdf)

* 2023-01    Hamilton: Natively bringing software engineering best practices to python data transformations. AI Camp Meetup San Jose
* [Slides](https:/skrawcz/talks/files/10830349/Hamilton_.Natively.bringing.software.engineering.best.practices.to.python.data.transformations.-.January.2023.pdf)

* 2022-10    Hamilton: an open source, declarative, micro-framework for clean & robust feature transform code in Python. Feature Store Summit
* [Event](https://www.featurestoresummit.com/)
* [Slides](https:/skrawcz/talks/files/9759661/FS.Summit.2022.-.Hamilton.pdf)

* 2022-09    Hamilton: enabling software engineering best practices for data transformations via generalized dataflow graphs. DEco - First International Workshop on Data Ecosystems
* [Event](https://dbis.rwth-aachen.de/DEco22/)
* [Slides](https:/skrawcz/talks/files/9550914/Submitted.-.DEco.2022_.Hamilton_.enabling.software.engineering.best.practices.for.data.transformations.via.generalized.dataflow.graphs.1.pdf)

* 2022-09    Hamilton: a modular open source declarative paradigm for high level modeling of dataflows. CDMS - First International Workshop on Composable Data Management Systems
* [Event](https://cdmsworkshop.github.io/2022/)
* [Slides](https:/skrawcz/talks/files/9550939/CDMS.2022.-.Hamilton_.a.modular.open.source.declarative.paradigm.for.high.level.modeling.of.dataflows.1.pdf)
* [Paper](https://cdmsworkshop.github.io/2022/Proceedings/ShortPapers/Paper6_StefanKrawczyk.pdf)

* 2022-08    Hamilton: A Python Micro-Framework for tidy scalable Pandas. Scalable Pandas Meetup
* [Recording](https://www.youtube.com/watch?v=m_rjCzxQj4c&ab_channel=Ponder)
* [Slides](https:/skrawcz/talks/files/9428705/Hamilton.%40.Ponder.Pandas.meetup.pdf)

* 2022-08    Scalable feature engineering with Hamilton on Ray. Ray Summit
* [Slides](https:/skrawcz/talks/files/9411082/Submitted.Slides.-.Ray.Summit_.Scalable.feature.engineering.with.Hamilton.on.Ray.pdf)

* 2022-07    Hamilton: A Python Micro-Framework for Data / Feature Engineering. MLOPsWorld Bay Area
* [Slides](https:/skrawcz/talks/files/9213924/Hamilton_.A.Python.Micro-Framework.for.Data._.Feature.Engineering.pdf)

* 2022-05    Hamilton: a python micro-framework for data / feature engineering at Stitch Fix. AICamp
* [Recording](https://www.youtube.com/watch?v=PDGIt37dov8)
* [Slides](https:/skrawcz/talks/files/8691633/AICamp.Hamilton.Presentation.pdf)

* 2022-02    [Open Source] Hamilton, a micro framework for creating dataframes, and its application at Stitch Fix. Apply(Meetup)
* [Event](https://www.applyconf.com/agenda/open-source-hamilton-a-micro-framework-for-creating-dataframes-and-its-application-at-stitch-fix/).
* [Recording](https://www.youtube.com/watch?v=CHfrT5OVjlM)
* [Slides](https:/skrawcz/talks/blob/main/Public%20ApplyConf2022%20-%20%5BOpen%20Source%5D%20Hamilton%2C%20a%20micro%20framework%20for%20creating%20dataframes%2C%20and%20its%20application%20at%20Stitch%20Fix.pdf)

* 2021-12    Hamilton an open source micro framework for creating dataframes. SF Python Meetup
* [Recording](https://www.youtube.com/watch?v=_XUYfwougz4)
* [Slides](https:/skrawcz/talks/files/8944605/Python.Meetup.Dec.2021.-.Hamilton_.an.open.source.micro.framework.for.creating.dataframes.pdf)


## 📰 External Blogs

* 2022-07    [Tidy production pandas with Hamilton](https://towardsdatascience.com/tidy-production-pandas-with-hamilton-3b759a2bf562)
* 2022-06    [Developing Scalable Feature Engineering DAGs with Metaflow & Hamilton](https://outerbounds.com/blog/developing-scalable-feature-engineering-dags)
* 2022-05    [Hamilton backstory and intro post on TDS](https://towardsdatascience.com/functions-dags-introducing-hamilton-a-microframework-for-dataframe-generation-more-8e34b84efc1d)
* 2022-05    [Hamilton + Pandas in five minutes](https://towardsdatascience.com/how-to-use-hamilton-with-pandas-in-5-minutes-89f63e5af8f5)
* 2022-05    [Iterating with Hamilton in a Notebook](https://towardsdatascience.com/how-to-use-hamilton-with-pandas-in-5-minutes-89f63e5af8f5)

## 🎙 Podcasts
* 2022-08    [S01 E08 - MLOps Week 8: The MLOps Mindset with Stefan Krawczyk](https://rss.com/podcasts/mlops-weekly/571949/)
* 2022-04    [MLOps dla 100 data scientistów](https://nieliniowy.pl/mlops-dla-100-data-scientistow-stefan-krawczyk-stitch-fix/) (in Polish)
* 2021-09    [Aggressively Helpful Platform teams](https://www.youtube.com/watch?v=az8lXG9v4uo)
2 changes: 1 addition & 1 deletion docs/license.rst → docs/get-started/license.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,6 @@ No user data or potentially sensitive information is or ever will be collected.
the name of the Driver function being invoked.


Else see :doc:`/reference/miscellaneous/disabling-telemetry` for how to disable telemetry.
Else see :doc:`/reference/disabling-telemetry` for how to disable telemetry.

Otherwise we invite you to inspect telemetry.py for details.
Loading