Add getting started docs and Postgres tutorial #258

mknycha · 2019-08-07T05:42:21Z

No description provided.

dhui

Thanks for taking a stab at this and opening the PR! This is a great start!

I think it'd be better to introduce concepts and best practices in the "Getting Started" doc and have DB specific examples in each DB.
It makes sense to rename TUTORIAL.md to GETTING_STARTED.md, move the postgres specific example/tutorial to databases/postgres/TUTORIAL.md, and create a table of contents for DB specific tutorials in GETTING_STARTED.md.

dhui · 2019-08-07T07:23:10Z

FAQ.md

@@ -68,3 +68,6 @@
 Database-specific locking features are used by *some* database drivers to prevent multiple instances of migrate from running migrations at the same time
 the same database at the same time. For example, the MySQL driver uses the `GET_LOCK` function, while the Postgres driver uses
 the `pg_advisory_lock` function.
+
+#### Do I need to create a table for tracking migration version used?
+No, it is done automatically. This information will be kept in the table `schema_migrations` in the database that you have configured.


The actual schema migrations table name is set by the DB driver. Generally, it's schema_migrations, but that name is not enforced. I haven't audited the names used by all the DB drivers, so I don't know how accurate this statement is. To be safe and more resilient to future changes, let's not make such an authoritative statement.

dhui · 2019-08-07T07:24:09Z

TUTORIAL.md

@@ -0,0 +1,100 @@
+# Getting started
+Before you start, make sure you understand the concept or database migration and rollback.


Rephrase to "make sure you understand the concept of forward/up and reverse/down database migrations"

dhui · 2019-08-07T07:25:52Z

TUTORIAL.md

+# Getting started
+Before you start, make sure you understand the concept or database migration and rollback.
+
+Create new Go project and configure a database for it. Make sure that your database is supported [here](README.md#databases)


Migrate can be used to manage migrations with non-Go projects via the CLI, so creating a Go project isn't necessary.

Fair enough. Is it worth adding it to FAQ?

My guess is that migrate is primarily used with Go projects today, but that doesn't have to be the case. No one has asked the question yet, but we could add it to the FAQ 😄

e.g.
Can I use migrate with a non-Go project?
Yes, you can use the migrate CLI in a non-Go project, but there are probably other libraries/frameworks available that offer better test and deploy integrations in that language/framework.

Let's assume I have asked it three times recently :D but honestly, I think it will not be harmful to add it there ;)

dhui · 2019-08-07T07:27:01Z

TUTORIAL.md

+
+Create new Go project and configure a database for it. Make sure that your database is supported [here](README.md#databases)
+For the purpose of this tutorial let's create PostgreSQL database called `example`.
+Our user here is `postgres`, and host is `localhost`.


Also mention what the password is. I'd use a password that's distinct from the username for clarity

dhui · 2019-08-07T07:29:10Z

TUTORIAL.md

+## Create migrations
+Let's create table called `users`:
+```
+migrate create -ext sql -dir db/migrations create_users_table


I'd recommend using the -seq option

I agree that it makes the example easier to understand but using -seq in a project used by many developers can go awry as they can push two migrations with the same sequential number. I feel like it's one of these "know what you're doing" options, while I wanted the tutorial to be accessible for someone who did not use database migrations before. What's your opinion here?

Detecting conflicting migrations is an unresolved issue. Using sequentially numbered vs timestamps doesn't fix the issue. e.g. when using timestamps, 2 different developers could create conflicting migrations and the developer who created created the migration second gets their PR merged before the first one.

Could you also mention this caveat (that conflicting migrations aren't detected) in our docs.

For more info, see: #179 (comment)

I have not thought about that. I will mention it in getting started docs.

dhui · 2019-08-07T07:30:01Z

TUTORIAL.md

+```
+migrate create -ext sql -dir db/migrations create_users_table
+```
+If there were no errors, we should have two files available under `db/migrations` folder. Note the `sql` extension that we provided.


Be explicit in the example and actually list the files that were created

dhui · 2019-08-07T07:33:28Z

TUTORIAL.md

+If there were no errors, we should have two files available under `db/migrations` folder. Note the `sql` extension that we provided.
+In the `.up.sql` file we are going to create table:
+```
+CREATE TABLE users(


Should also mention idempotency in migrations and use that in the examples. e.g. IF NOT EXISTS and IF EXISTS and wrapping your migration in a transaction if DDL transactions are supported by your DB.

dhui · 2019-08-07T07:40:14Z

TUTORIAL.md

+(e.g. if you created a table in a migration but rollback did not delete it, you will encounter an error when running the migration again)
+It's also worth checking your migrations in a separate, containerized environment. You can find some tools in the end of this tutorial.
+
+**Hint:** Most probably you are going to use to the above commands often, it's worth putting them in a Makefile of your project.


Could you provide more background on why you found wrapping the CLI usage in a Makefile useful?
I've been using the CLI directly.

In our project at work, we use it to call make migrate-up POSTGRESQL_DB=example_test steps=2 instead of migrate -database postgres://user:password@localhost:5432/example_test -path ./migrations/postgres up 2. It's just slightly shorter compared to putting database URL in .env file.

Ahhh, I normally have my Go apps use migrate as a library and call migrate.Migrate.Up() before serving traffic. Managing migrations out-of-app using the CLI makes sense too. e.g. there are many different dev and deploy processes
I wouldn't assume that all projects are using make to manage their development and/or deploys.

Right, it may be too much environment-specific. I will remove this part

dhui · 2019-08-07T07:41:29Z

TUTORIAL.md

+```
+You can find details [here](README.md#use-in-your-go-project)
+
+Just add the code to your app and you're ready to go!


Add a caveat about not all DBs supporting locking when running migrations, so if you run multiple instances of your app on different machines, you may encounter issues.

dhui · 2019-08-07T07:43:56Z

Addresses: #40

And add migration conflicts description

As it is too much environment-specific

mknycha · 2019-08-09T20:18:29Z

Just wanted to let know that I will reorganize files during the weekend.

dhui

Wow, this looks great! Only a few minor quibbles left.

dhui · 2019-08-13T05:58:06Z

GETTING_STARTED.md

+```
+Once you create your files, you should fill them.
+
+**IMPORTANT:** In a project developed by more than one person there is a small probability of migrations incosistency - e.g. two developers can create conflicting migrations, and the developer that created his migration later gets it merged to the repository first.


Remove the word "small" since the chances increase with team size.

dhui · 2019-08-13T06:05:51Z

GETTING_STARTED.md

+Once you create your files, you should fill them.
+
+**IMPORTANT:** In a project developed by more than one person there is a small probability of migrations incosistency - e.g. two developers can create conflicting migrations, and the developer that created his migration later gets it merged to the repository first.
+Keep an eye on such cases (and be even more careful when cherry picking).


Remove reference to cherry picking since that doesn't increase the likelihood of an inconsistency occurring. I'd just note that developers and teams should keep any eye out for this issue, especially during code review. A note or link to this summary would be nice too. Namely it'd be good to get more input on the what's the best solution for this problem.

Ok. What I meant here is that someone could cherry pick changes including migration dependent on some earlier migrations.

dhui · 2019-08-13T15:30:49Z

Thanks for improving our docs! This should make it easier for new users to get started and help clarify some points/usage for existing users.

Marcin K added 4 commits August 7, 2019 01:00

Add tutorial file

01107cb

Add reference to README

74b9fdb

FAQ: Add one more question

d7e0e40

Tutorial: Remove info about migration version table

1d402e4

dhui requested changes Aug 7, 2019

View reviewed changes

Marcin K added 11 commits August 7, 2019 21:52

FAQ: Remove driver specific info

e98f9f6

Use reverse/down migration instead of rollback

44b07c5

Rephrase, list created files

fb771c8

TUTORIAL: Use password explicitly

b49854b

Use sequential migrations naming in example

604f125

And add migration conflicts description

Use idempotent queries

fb4fb97

Add a caveat regarding DB lock when running migrations

d6fa9be

Cosmetics

d90f7d5

Describe database transactions

bb1e9dd

Remove hint regarding Makefile

d0c2e44

As it is too much environment-specific

FAQ: Add question regarding non-Go project usage

86e4e49

Put tutorial specific info into a separate file

12f7e22

mknycha changed the title ~~Add tutorial for getting started~~ Add getting started docs and Postgres tutorial Aug 12, 2019

dhui requested changes Aug 13, 2019

View reviewed changes

Marcin K added 2 commits August 13, 2019 16:38

Update info regarding migrations inconsistency

98c383c

Use different word

d3ec78d

dhui approved these changes Aug 13, 2019

View reviewed changes

dhui merged commit b071731 into golang-migrate:master Aug 13, 2019

dhui mentioned this pull request Aug 13, 2019

Add an example for beginners #40

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add getting started docs and Postgres tutorial #258

Add getting started docs and Postgres tutorial #258

mknycha commented Aug 7, 2019

dhui left a comment

dhui Aug 7, 2019

mknycha Aug 7, 2019

dhui Aug 7, 2019

mknycha Aug 7, 2019

dhui Aug 7, 2019

mknycha Aug 7, 2019

dhui Aug 8, 2019

mknycha Aug 8, 2019 •

edited

Loading

dhui Aug 7, 2019

mknycha Aug 7, 2019

dhui Aug 7, 2019

mknycha Aug 7, 2019

dhui Aug 8, 2019

mknycha Aug 8, 2019

dhui Aug 7, 2019

mknycha Aug 7, 2019

dhui Aug 7, 2019

mknycha Aug 8, 2019

dhui Aug 7, 2019

mknycha Aug 8, 2019

dhui Aug 9, 2019

mknycha Aug 9, 2019 •

edited

Loading

dhui Aug 7, 2019

mknycha Aug 8, 2019

dhui commented Aug 7, 2019

mknycha commented Aug 9, 2019

dhui left a comment

dhui Aug 13, 2019

mknycha Aug 13, 2019

dhui Aug 13, 2019

mknycha Aug 13, 2019

dhui commented Aug 13, 2019

		@@ -0,0 +1,100 @@
		# Getting started
		Before you start, make sure you understand the concept or database migration and rollback.

Add getting started docs and Postgres tutorial #258

Add getting started docs and Postgres tutorial #258

Conversation

mknycha commented Aug 7, 2019

dhui left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mknycha Aug 8, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mknycha Aug 9, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dhui commented Aug 7, 2019

mknycha commented Aug 9, 2019

dhui left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dhui commented Aug 13, 2019

mknycha Aug 8, 2019 •

edited

Loading

mknycha Aug 9, 2019 •

edited

Loading