Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apply dir overrides from agent config #894

Merged
merged 9 commits into from
Nov 16, 2023
Merged

Conversation

ofalvai
Copy link
Contributor

@ofalvai ofalvai commented Nov 13, 2023

Checklist

Version

Requires a MINOR* version update

Context

Follow-up to #892, this PR is about applying the defined Bitrise dir overrides when running a workflow

Changes

  • Override Bitrise dir env vars if an agent config file is found
  • Make workflow run logging a bit prettier and also log if it's running in agent mode
  • Remove ASCII art logging

Investigation details

Decisions

Applying the other configs from agent-config.yml is out of scope for this PR

log/log.go Show resolved Hide resolved
log/log.go Outdated
m.Print("")
m.Printf("Invocation started at %s", colorstring.Cyan(m.opts.TimeProvider().Format(consoleTimeLayout)))
m.Printf("Bitrise CLI version: %s", colorstring.Cyan(plan.Version))
m.Print("")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: you can save some time by omitting the "" parameter to print a newline. It will work with an empty argument because the logger will append a \n char to whatever you supply here. And supplying nothing will end up with juts the single \n char.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, TIL

if err != nil {
return fmt.Errorf("agent config file: %w", err)
}
log.Printf("env: %s", os.Getenv(BitriseDataHomeDirEnvKey))
Copy link
Contributor

@tothszabi tothszabi Nov 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a leftover print statement? Just asking because when it is executed it will print out something like

env: /path/to/some/folder

and if the folder does not have a clear name then (without knowing the CLI codebase) it will be hard to understand for the users what env is being printed out.

So I think we should either add the env name into the print statement or remove it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it was just a leftover debug statement

if err != nil {
return fmt.Errorf("set BITRISE_DATA_HOME_DIR: %w", err)
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super nit: there are 2 newlines here while everywhere else in the same file there is only 1.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

log/log.go Outdated
Comment on lines 154 to 156
if plan.DebugMode {
m.Printf("Debug mode: %v", colorstring.Cyan("%v", plan.DebugMode))
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think it is perfectly fine to always print out if we are in debug mode or not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good point, if for nothing else, this introduces people to the debug flag 😆

@@ -53,6 +55,72 @@ type AgentHooks struct {
DoOnWorkflowEnd string `yaml:"do_on_workflow_end"`
}

func RegisterAgentOverrides() error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to closely look at and compare all of 4 sections of the dir handling sections inside this function to discover that they are exactly the same. This made me wonder if it would be good to group them and create a single logic with multiple input parameters for them to indicate that they are the same.

Something like

params := []struct {
	name   string
	dir    string
	envKey string
}{
	{
		name:   "BITRISE_DATA_HOME_DIR",
		dir:    config.BitriseDirs.BitriseDataHomeDir,
		envKey: BitriseDataHomeDirEnvKey,
	},
	{
		name:   "BITRISE_SOURCE_DIR",
		dir:    config.BitriseDirs.SourceDir,
		envKey: BitriseSourceDirEnvKey,
	},
	{
		name:   "BITRISE_DEPLOY_DIR",
		dir:    config.BitriseDirs.DeployDir,
		envKey: BitriseDeployDirEnvKey,
	},
	{
		name:   "BITRISE_TEST_DEPLOY_DIR",
		dir:    config.BitriseDirs.TestDeployDir,
		envKey: BitriseTestDeployDirEnvKey,
	},
}
for _, param := range params {
	err = pathutil.EnsureDirExist(param.dir)
	if err != nil {
		return fmt.Errorf("can't create %s: %w", param.name, err)
	}
	err = os.Setenv(param.envKey, param.dir)
	if err != nil {
		return fmt.Errorf("set %s: %w", param.name, err)
	}
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, I fully agree, please take a look now

Comment on lines +225 to +227
if err := configs.RegisterAgentOverrides(); err != nil {
return models.BuildRunResultsModel{}, fmt.Errorf("apply agent config: %s", err)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now this checks happens in the workflow execution process but to me this seems like not directly linked to it. If we can then I would put this somewhere earlier in the chain. Like as high level as we can put it. Maybe before this line:

runner := NewWorkflowRunner(*config)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, my mental model is that all of these agent-specific modifications are only relevant to the bitrise run subcommand. Do you think it should apply to other subcommands too?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree with you that this is bitrise run specific. The code I linked is in the run.go file which is hosting the bitrise run subcommand:

bitrise/cli/run.go

Lines 50 to 71 in cbfa1bb

var runCommand = cli.Command{
Name: "run",
Aliases: []string{"r"},
Usage: "Runs a specified Workflow.",
Action: run,
Flags: []cli.Flag{
// cli params
cli.StringFlag{Name: WorkflowKey, Usage: "workflow id to run."},
cli.StringFlag{Name: ConfigKey + ", " + configShortKey, Usage: "Path where the workflow config file is located."},
cli.StringFlag{Name: InventoryKey + ", " + inventoryShortKey, Usage: "Path of the inventory file."},
cli.BoolFlag{Name: secretFilteringFlag, Usage: "Hide secret values from the log."},
// cli params used in CI mode
cli.StringFlag{Name: JSONParamsKey, Usage: "Specify command flags with json string-string hash."},
cli.StringFlag{Name: JSONParamsBase64Key, Usage: "Specify command flags with base64 encoded json string-string hash."},
cli.StringFlag{Name: OutputFormatKey, Usage: "Log format. Available values: json, console"},
// should deprecate
cli.StringFlag{Name: ConfigBase64Key, Usage: "base64 encoded config data."},
cli.StringFlag{Name: InventoryBase64Key, Usage: "base64 encoded inventory data."},
},
}

What I was referring to is that where this logic is now is a place where the actual execution flow is calculated inside the WorkflowRunner struct. This agent dir override seemed like something this runner should not be even aware and that is why I thought moving it out to the very beginning of the run subcommand execution could make sense.

@ofalvai ofalvai merged commit b3b017c into master Nov 16, 2023
6 checks passed
@ofalvai ofalvai deleted the BE-1248-apply-agent-config branch November 16, 2023 09:42
@ofalvai ofalvai mentioned this pull request Nov 20, 2023
2 tasks
@godrei godrei mentioned this pull request Nov 30, 2023
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants