From 19bc8d10f5569a39928c8e0f4ddaa73e4c1ffa8e Mon Sep 17 00:00:00 2001 From: Daniele Scasciafratte Date: Thu, 4 Jul 2024 16:59:14 +0200 Subject: [PATCH 1/9] feat(contribute): update for tests --- CONTRIBUTING.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fb5bafc8..64c26eac 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -104,7 +104,12 @@ Basically, the `translate()` method should return a `String` for the compiler to ### 4. Tests Amber uses `cargo test` for tests. `stdlib` and `validity` tests usually work by executing amber code and checking its output. -We have [`validity tests`](src/tests/validity.rs) to check the compiler, [`stdlib tests`](src/tests/stdlib.rs) and [`CLI tests`](src/tests/cli.rs). +We have [`validity tests`](src/tests/validity.rs) to check the compiler, [`stdlib tests`](src/tests/stdlib.rs) and [`CLI tests`](src/tests/cli.rs). + +The majority of `stdlib` tests are Written in pure Amber in the folder [`tests/stdlib`](src/tests/stdlib), for every test there is a `*.output.txt` file. +Any test will be executed without the need to compile again Amber, will load the Amber scripts and check for the output in the dedicated file to pass the test. + +A part of those tests like for `download` require Rust to load a web server, so there is another folder [`tests/stdlib/no_output`](src/tests/stdlib/no_output) that include just the Amber script, the output is inside the [`stdlib tests`](src/tests/stdlib.rs) file.
Let's write a simple test From bdfcd0e40566eb94f9ed54e6fbd64694c14e6e96 Mon Sep 17 00:00:00 2001 From: Daniele Scasciafratte Date: Fri, 5 Jul 2024 10:21:27 +0200 Subject: [PATCH 2/9] Update CONTRIBUTING.md Co-authored-by: Phoenix Himself --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 64c26eac..0a3c2967 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -106,7 +106,7 @@ Amber uses `cargo test` for tests. `stdlib` and `validity` tests usually work by We have [`validity tests`](src/tests/validity.rs) to check the compiler, [`stdlib tests`](src/tests/stdlib.rs) and [`CLI tests`](src/tests/cli.rs). -The majority of `stdlib` tests are Written in pure Amber in the folder [`tests/stdlib`](src/tests/stdlib), for every test there is a `*.output.txt` file. +The majority of `stdlib` tests are Written in pure Amber in the folder [`tests/stdlib`](src/tests/stdlib). For every test there is a `*.output.txt` file that contains the expected output. Any test will be executed without the need to compile again Amber, will load the Amber scripts and check for the output in the dedicated file to pass the test. A part of those tests like for `download` require Rust to load a web server, so there is another folder [`tests/stdlib/no_output`](src/tests/stdlib/no_output) that include just the Amber script, the output is inside the [`stdlib tests`](src/tests/stdlib.rs) file. From a1d012b2997a107ebfbf0a21e719ed40962bdaed Mon Sep 17 00:00:00 2001 From: Daniele Scasciafratte Date: Fri, 5 Jul 2024 10:21:33 +0200 Subject: [PATCH 3/9] Update CONTRIBUTING.md Co-authored-by: Phoenix Himself --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0a3c2967..4391d422 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -107,7 +107,7 @@ Amber uses `cargo test` for tests. `stdlib` and `validity` tests usually work by We have [`validity tests`](src/tests/validity.rs) to check the compiler, [`stdlib tests`](src/tests/stdlib.rs) and [`CLI tests`](src/tests/cli.rs). The majority of `stdlib` tests are Written in pure Amber in the folder [`tests/stdlib`](src/tests/stdlib). For every test there is a `*.output.txt` file that contains the expected output. -Any test will be executed without the need to compile again Amber, will load the Amber scripts and check for the output in the dedicated file to pass the test. +Tests will be executed without recompilation. Amber will load the scripts and verify the output in the designated file to determine if the test passes. A part of those tests like for `download` require Rust to load a web server, so there is another folder [`tests/stdlib/no_output`](src/tests/stdlib/no_output) that include just the Amber script, the output is inside the [`stdlib tests`](src/tests/stdlib.rs) file. From f3860820896235993540c3340d6572bce272a4bf Mon Sep 17 00:00:00 2001 From: Daniele Scasciafratte Date: Fri, 5 Jul 2024 10:21:38 +0200 Subject: [PATCH 4/9] Update CONTRIBUTING.md Co-authored-by: Phoenix Himself --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4391d422..fd5d4aa0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -104,7 +104,7 @@ Basically, the `translate()` method should return a `String` for the compiler to ### 4. Tests Amber uses `cargo test` for tests. `stdlib` and `validity` tests usually work by executing amber code and checking its output. -We have [`validity tests`](src/tests/validity.rs) to check the compiler, [`stdlib tests`](src/tests/stdlib.rs) and [`CLI tests`](src/tests/cli.rs). +We have [`validity tests`](src/tests/validity.rs) to check if the compiler outputs a valid bash code, [`stdlib tests`](src/tests/stdlib.rs) and [`CLI tests`](src/tests/cli.rs). The majority of `stdlib` tests are Written in pure Amber in the folder [`tests/stdlib`](src/tests/stdlib). For every test there is a `*.output.txt` file that contains the expected output. Tests will be executed without recompilation. Amber will load the scripts and verify the output in the designated file to determine if the test passes. From 9d8b39cb649a0d21e4de31d6166a9a9c240250f1 Mon Sep 17 00:00:00 2001 From: Daniele Scasciafratte Date: Fri, 5 Jul 2024 10:21:47 +0200 Subject: [PATCH 5/9] Update CONTRIBUTING.md Co-authored-by: Phoenix Himself --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fd5d4aa0..192f52bc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -109,7 +109,7 @@ We have [`validity tests`](src/tests/validity.rs) to check if the compiler outpu The majority of `stdlib` tests are Written in pure Amber in the folder [`tests/stdlib`](src/tests/stdlib). For every test there is a `*.output.txt` file that contains the expected output. Tests will be executed without recompilation. Amber will load the scripts and verify the output in the designated file to determine if the test passes. -A part of those tests like for `download` require Rust to load a web server, so there is another folder [`tests/stdlib/no_output`](src/tests/stdlib/no_output) that include just the Amber script, the output is inside the [`stdlib tests`](src/tests/stdlib.rs) file. +Some tests, such as those for `download`, require Rust to load a web server. Therefore, there is a separate folder [`tests/stdlib/no_output`](src/tests/stdlib/no_output) that contains only the Amber script. The output is located in the [`stdlib tests`](src/tests/stdlib.rs) file.
Let's write a simple test From 8fea601b57d8a3a8cb5c47ab7e71261a44fff290 Mon Sep 17 00:00:00 2001 From: Daniele Scasciafratte Date: Wed, 10 Jul 2024 11:10:52 +0200 Subject: [PATCH 6/9] Update CONTRIBUTING.md Co-authored-by: Phoenix Himself --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 192f52bc..9dbb5fa8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -109,7 +109,7 @@ We have [`validity tests`](src/tests/validity.rs) to check if the compiler outpu The majority of `stdlib` tests are Written in pure Amber in the folder [`tests/stdlib`](src/tests/stdlib). For every test there is a `*.output.txt` file that contains the expected output. Tests will be executed without recompilation. Amber will load the scripts and verify the output in the designated file to determine if the test passes. -Some tests, such as those for `download`, require Rust to load a web server. Therefore, there is a separate folder [`tests/stdlib/no_output`](src/tests/stdlib/no_output) that contains only the Amber script. The output is located in the [`stdlib tests`](src/tests/stdlib.rs) file. +Some tests require additional setup, such as those for `download` that needs Rust to load a web server. These functions require special tests written in Rust that we can find in [`stdlib tests`](src/tests/stdlib.rs) file. The designated directory where to store the amber files is located in [`tests/stdlib/no_output`](src/tests/stdlib/no_output). These tests do not coexist with `.output.txt` files hence the name of this folder.
Let's write a simple test From 8ca04faafd791218297b9ce9ec79b1be997bcd72 Mon Sep 17 00:00:00 2001 From: Daniele Scasciafratte Date: Wed, 10 Jul 2024 11:13:22 +0200 Subject: [PATCH 7/9] builtin From https://github.com/amber-lang/amber/issues/157#issuecomment-2218331117 --- CONTRIBUTING.md | 104 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9dbb5fa8..f7702a42 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -30,6 +30,7 @@ Amber consists of the following layers: 2. [Compiler](#2-compiler) 1. [Parser & tokenizer](#21-parser--tokenizer) 2. [Translator](#22-translator) + 2. [Built-in](#23-built-in-creation) 3. [Runtime libraries](#3-runtime-libraries) 1. [`stdlib`](#31-stdlib) 4. [Tests](#4-tests) @@ -96,6 +97,109 @@ fn translate() -> String { Basically, the `translate()` method should return a `String` for the compiler to construct a compiled file from all of them. If it translates to nothing, you should output an empty string, like `String::new()` +#### 2.3. Built-in creation + +In this guide we will see how to create a basic built-in function that in Amber syntax presents like: +```sh +builtin "Hello World" +``` +And compiles to: +```sh +echo "Hello World" +``` + +Let's create a `src/modules/builtin/builtin.rs` file with the following content: + +```rs +// This is the prelude that imports all necessary stuff of Heraclitus framework for parsing the syntax +use heraclitus_compiler::prelude::*; +// Expression module that can parse expressions +use crate::modules::expression::expr::Expr; +// Translate module is not included in Heraclitus prelude as it's leaving the backend up to developer +use crate::translate::module::TranslateModule; +// Metadata is the object that is carried when iterating over syntax tree. +// - `ParserMetadata` - it carries the necessary information about the current parsing context such as variables and functions that were declared up to this point, warning messages aggregated up to this point, information whether this syntax is declared in a loop, function, main block, unsafe scope etc. +// `TranslateMetadata` - it carries the necessary information for translation such as wether we are in a silent scope, in an eval context or what indentation should be used. +use crate::utils::{ParserMetadata, TranslateMetadata}; + +// This is a declaration of your built-in. Set the name accordingly. +#[derive(Debug, Clone)] +pub struct Builtin { + // This particular built-in contains a single expression + value: Expr +} + +// This is an implementation of a trait that creates a parser for this module +impl SyntaxModule for Echo { + // Here you can define the name of this built-in that will displayed when debugging the parser + syntax_name!("Builtin"); + + // This function should always contain the default state of this syntax module + fn new() -> Self { + Echo { + value: Expr::new() + } + } + + // This is a function that will parse this syntax module "Built-in". It returns SyntaxResult which is a `Result<(), Failure>` where the `Failure` is an Heraclitus primitive that returns an error. It can be either: + - `Quiet` - which means that this is not the right syntax module to parse + - `Loud` - which means that this is the correct syntax module but there is some critical error in the code that halts the entire compilation process + fn parse(&mut self, meta: &mut ParserMetadata) -> SyntaxResult { + // `token` parses a token `builtin` which is basically a command name for our built-in. + // If we add `?` in the end of the heraclitus provided function - this function will return a quiet error. + token(meta, "builtin")?; + // `syntax` parses the `Expr` expression syntax module + syntax(meta, &mut self.value)?; + // This terminates parsing process with success exit code + Ok(()) + } +} + +// Here we implement the translator for the syntax module. Here we return valid Bash or sh code. +impl TranslateModule for Builtin { + // Here we define the valid translate function. The String returns the current line. + fn translate(&self, meta: &mut TranslateMetadata) -> String { + // Here we run the translate function on the syntax module `Expr` + let value = self.value.translate(meta); + // Here we return the Bash code + format!("echo {}", value) + } +} +``` + +Now let's import it in the main module for built-ins `src/modules/builtin/mod.rs` + +```rs +pub mod echo; +pub mod nameof; +// ... +pub mod builtin; +``` + +Now we have to integrate this syntax module with either statement `Stmt` or expression `Expr`. Since this is a statement module, we'll add it to the list of statement syntax modules. Let's modify `src/modules/statement/stmt.rs`: + +```rs +// Let's import it first +use crate::modules::builtin::builtin::Builtin; + +// Let's add it to the statement type enum +pub enum StatementType { + // ... + Builtin(Builtin) +} + +// Now, let's add it to the list of statement syntax modules, arranged in the order of parsing precedence: +impl Statement { + handle_types!(StatementType, [ + // ... + Builtin, + // ... + } + + // ... +} +``` + ### 3. Runtime libraries #### 3.1. `stdlib` From 2e9ec5d3714e3b4dfb604b424d989da2f11f62c5 Mon Sep 17 00:00:00 2001 From: Daniele Scasciafratte Date: Thu, 11 Jul 2024 10:34:19 +0200 Subject: [PATCH 8/9] feat(review): added details --- CONTRIBUTING.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f7702a42..5a470121 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -108,8 +108,9 @@ And compiles to: echo "Hello World" ``` +
+Let's start! Let's create a `src/modules/builtin/builtin.rs` file with the following content: - ```rs // This is the prelude that imports all necessary stuff of Heraclitus framework for parsing the syntax use heraclitus_compiler::prelude::*; @@ -199,6 +200,7 @@ impl Statement { // ... } ``` +
### 3. Runtime libraries #### 3.1. `stdlib` From 35cd476899d9dc3649e3ab54ee60fa2660652ed0 Mon Sep 17 00:00:00 2001 From: Daniele Scasciafratte Date: Thu, 11 Jul 2024 10:35:35 +0200 Subject: [PATCH 9/9] Update CONTRIBUTING.md --- CONTRIBUTING.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5a470121..b9651489 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -110,7 +110,10 @@ echo "Hello World"
Let's start! -Let's create a `src/modules/builtin/builtin.rs` file with the following content: + +Create a `src/modules/builtin/builtin.rs` file with the following content: + + ```rs // This is the prelude that imports all necessary stuff of Heraclitus framework for parsing the syntax use heraclitus_compiler::prelude::*;