Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support for structured scaffold in create command #970

Open
wants to merge 66 commits into
base: main
Choose a base branch
from

Conversation

ashupednekar
Copy link

Description

This PR fixes #969

Summary

PR: Add Structured Scaffold Option to Robyn's Create Command

Description:

This PR introduces a new feature to Robyn's create command, allowing users to choose between two scaffold options:

  1. Simple Starter Kit – Ideal for minimal, quick-start projects.
  2. Structured Scaffold – A more opinionated setup, providing separation of concerns and enhanced organization for scaling projects.

Changes:

  • Updated the create command to prompt users with a new option:
    • Would you like the scaffold to be a simple starter kit or an opinionated structure?
  • The simple scaffold remains the same as before with options like Mongo, Postgres, SQLite, etc.
  • Added a structured scaffold with the following structure:
    • api/handlers (for route handlers)
    • middlewares (for middleware definitions)
    • adaptors (with models, selectors, mutators)
    • utils (helper functions like db.py)
    • devops (including Dockerfile, docker-compose)
    • Improved configuration management (e.g., conf.py, config.env).

Example Structured Scaffold:

├── no-db
│   ├── api
│   │   ├── handlers
│   │   │   ├── __init__.py
│   │   │   ├── probes.py
│   │   │   └── sample.py
│   │   └── middlewares
│   │       └── __init__.py
│   ├── conf.py
│   ├── config.env
│   ├── devops
│   │   ├── Dockerfile
│   │   ├── Dockerfile.src
│   │   └── docker-compose.yaml
│   ├── requirements.txt
│   ├── server.py
│   └── utils
│       └── __init__.py
└── sqlalchemy
    ├── adaptors
    │   ├── __init__.py
    │   ├── models.py
    │   ├── mutators
    │   │   └── __init__.py
    │   ├── schema.py
    │   └── selectors
    │       └── __init__.py
    ├── api
    │   ├── handlers
    │   │   ├── __init__.py
    │   │   ├── probes.py
    │   │   └── sample.py
    │   └── middlewares
    │       └── __init__.py
    ├── conf.py
    ├── config.env
    ├── devops
    │   ├── Dockerfile
    │   ├── Dockerfile.src
    │   └── docker-compose.yaml
    ├── requirements.txt
    ├── server.py
    └── utils
        ├── __init__.py
        └── db.py

PR Checklist

Please ensure that:

  • [*] The PR contains a descriptive title
  • [*] The PR contains a descriptive summary of the changes
  • [*] You build and test your changes before submitting a PR.
  • You have added relevant documentation
  • You have added relevant tests. We prefer integration tests wherever possible

Pre-Commit Instructions:

Copy link

vercel bot commented Sep 25, 2024

@pre-commit-ci[bot] is attempting to deploy a commit to the sparckles Team on Vercel.

A member of the Team first needs to authorize it.

@ashupednekar ashupednekar marked this pull request as draft September 25, 2024 17:29
Copy link

codspeed-hq bot commented Sep 25, 2024

CodSpeed Performance Report

Merging #970 will not alter performance

Comparing ashupednekar:feat_structured_scaffold (662c009) with main (06c487d)

Summary

✅ 116 untouched benchmarks

@ashupednekar ashupednekar changed the title Feat structured scaffold feat: support for structured scaffold in create command Sep 26, 2024
@ashupednekar ashupednekar marked this pull request as ready for review September 26, 2024 05:00
Copy link
Member

@sansyrox sansyrox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have some follow up suggestions

Comment on lines +110 to +113
├── migrations
│   ├── README
│   ├── env.py
│   ├── script.py.mako
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to read more about this

Comment on lines +15 to +16
with patch("robyn.cli.os.makedirs") as mock_makedirs:
with patch("robyn.cli.shutil.copytree") as mock_copytree, patch("robyn.os.remove") as _mock_remove:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not move this to line 8 ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't follow...

added additional key in mock return value scaffold_type
and added two tests for the two possible values

Comment on lines +6 to +12
@router.get("/livez/")
def livez() -> str:
return "live"


@router.get("/healthz/")
def healthz() -> str:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use more formal names here like

subrouter_healthcheck and subrouter_livecheck

Comment on lines 6 to 17
class SampleHandlers:
"""
note: the handles being grouped in a class like this is complete optional, and doesn't have any impact on routing
"""

@router.post("/one")
@staticmethod
def one(): ...

@router.get("/two")
@staticmethod
def two(): ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a big fan of this syntax honestly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we please change this

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should I remove the sample handlers class? it doesn't really serve any purpose as such.

Comment on lines +4 to +6
from robyn.helpers import discover_routes

app: Robyn = discover_routes("api.handlers")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where is this present?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

robyn/helpers.py

? Directory Path: myproject
? Need Docker? (Y/N) Y
? Please choose if you'd like the scaffold to be a simple starter kit or an opinionated structure
Simple
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ashupednekar , I don't see an update here

COPY . .
RUN pip install --no-cache-dir --upgrade -r requirements.txt --target=/workspace/deps

FROM gcr.io/distroless/python3-debian11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you explain a bit more here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a two stage commit...

  • takes python base image as builder
  • copies source and does pip install in this stage
  • second stage is gcr distroless debian 11 (https:/GoogleContainerTools/distroless)
  • distroless is a minimal image meant for production usage which is regularly maintained and patched for security vulnerabilities, and comes with an added benefit of small size as it doesn't include stuff from base linux image that's not needed at runtime
  • we copy the site-packages dir, which is /workspace/deps in our case so that our dependencies are available
  • finally, we set pythonpath and add server.py to the image's CMD

Copy link
Member

@sansyrox sansyrox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have some follow up suggestions

Comment on lines +132 to +138
from robyn.helpers import discover_routes
from robyn import Robyn

from utils.db import get_pool
from conf import settings

app: Robyn = discover_routes("api.handlers")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't have a discover_routes function

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's added in helpers as part of this PR...

def discover_routes(handler_path: str = "api.handlers") -> Robyn:
    mux: Robyn = Robyn(__file__)
    package = importlib.import_module(handler_path)
    for _, module_name, _ in pkgutil.iter_modules(package.__path__, package.__name__ + "."):
        module = importlib.import_module(module_name)
        logger.info(f"member: {module}")
        mux.include_router(module.router)
    return mux

Comment on lines +1 to +3
from .user import Base as user

metadata = [user.metadata]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a standard pattern?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, https://alembic.sqlalchemy.org/en/latest/autogenerate.html

alembic expects the model's sqlalchemy metadata to be passed in env.py
having it in models/init.py for better structure

which is used in alembic/env.py like so

from adaptors import models
target_metadata = models.metadata

Comment on lines +35 to +36
class Config:
orm_mode = True
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we dedent it?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the config? nope.... it's pydantic model config, it's supposed to be defined inline in the class definition
https://docs.pydantic.dev/1.10/usage/model_config/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you mention it in the code please?

Comment on lines +109 to +121
level = WARN
handlers = console
qualname =

[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine

[logger_alembic]
level = INFO
handlers =
qualname = alembic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do the empty assignments mean here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are alembic defaults, generated with alemic init migrations

The reason I chose to pre-run this and not leave it to users is so that we can make `migrations/env.py' changes to make is usable right away

Also, to standardize migrations directory structure

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ashupednekar , since alembic is not a tool where I have direct experience. Could you add a readme?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I add it in the scaffold, or should I add more docs ?

I've added what's needed to get started under example_app/index.mdx

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, added readme

# are written from script.py.mako
# output_encoding = utf-8

sqlalchemy.url = driver://user:pass@localhost/dbname
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let us leave a comment, which says that you need to update it accordingly.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Also, there was a bug in the online migrations function in alembic/env.py in structured sqlalchemy scaffold. Fixed, and added docs in example_app/index.mdx for database migration steps as well

COPY . .
RUN pip install --no-cache-dir --upgrade -r requirements.txt --target=/workspace/deps

FROM gcr.io/distroless/python3-debian11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you explain this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the --target flag tells pip to place the site packages at the specified path, we then copy this in the second stage and set PYTHONPATH accordingly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for structured scaffolding in create command, retaining current templates
4 participants