Broken emoji support

[elbrujohalcon]

Environment

• Add the result of rebar3 report to your message:

$ rebar3 report
===> Analyzing applications...
===> Compiling beamoji
Rebar3 report
 version 3.17.0
 generated at 2021-09-18T05:31:25+00:00
=================
Please submit this along with your issue at https:/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task:
Entered as:

-----------------
Operating System: x86_64-apple-darwin20.4.0
ERTS: Erlang/OTP 23 [erts-11.2.1] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [hipe]
Root Directory: /Users/fernandobenavides/.kerl/installations/23.3.2
Library directory: /Users/fernandobenavides/.kerl/installations/23.3.2/lib
-----------------
Loaded Applications:
bbmustache: 1.10.0
certifi: 2.6.1
cf: 0.3.1
common_test: 1.20.1
compiler: 7.6.7
crypto: 4.9
cth_readable: 1.5.1
dialyzer: 4.3.1
edoc: 0.12
erlware_commons: 1.5.0
eunit: 2.6
eunit_formatters: 0.5.0
getopt: 1.0.1
hipe: 4.0.1
inets: 7.3.2
kernel: 7.3
providers: 1.8.1
public_key: 1.10
relx: 4.5.0
sasl: 4.0.2
snmp: 5.8
ssl_verify_fun: 1.1.6
stdlib: 3.14.2
syntax_tools: 2.5
tools: 3.4.4

-----------------
Escript path: /usr/local/bin/rebar3
Providers:
  app_discovery as clean compile compile cover ct deps dialyzer do edoc emojify escriptize eunit get-deps hank help install install_deps kiwf list lock new path pkgs release relup report repos shell state tar tree unlock update upgrade upgrade upgrade version xref

Current behaviour

We're trying to create a plugin to run rebar3 🪄 and, in the process, we're spotting some issues related to emoji support.
This is our barebones version:

%%% @private
-module(beamoji_prv).

-export([init/1, do/1, format_error/1]).

%% @private
-spec init(rebar_state:t()) -> {ok, rebar_state:t()}.
init(State) ->
    Provider =
        providers:create([{name, '🪄'},
                          {module, beamoji_prv},
                          {bare, true},
                          {deps, []},
                          {example, "rebar3 🪄"},
                          {opts, []},
                          {short_desc, "🪄 ⇢ 🤯"},
                          {desc, "🪄📜 ⇢ 📃✨ ⇢ 🤯"}]),
    {ok, rebar_state:add_provider(State, Provider)}.

%% @private
%% @todo Implement this function
-spec do(rebar_state:t()) -> {ok, rebar_state:t()}.
do(State) ->
    rebar_api:warn("🪄 called ⚠️", []),
    {ok, State}.

%% @private
-spec format_error(any()) -> binary().
format_error(Reason) ->
    unicode:characters_to_binary(
        io_lib:format("~tp", [Reason])).

In this scenario, if we run rebar3 help on a project using the plugin…

$ DIAGNOSTIC=1 rebar3 help
===> Load global config file /Users/fernandobenavides/.config/rebar3/rebar.config
===> 23.3.2 satisfies the requirement for minimum OTP version 21
===> Setting paths to [deps]
===> Compile (apps)
===> Setting paths to [plugins]
===> Setting paths to [deps]
===> Setting paths to [plugins]
===> Setting paths to [plugins]
===> 23.3.2 satisfies the requirement for minimum OTP version 21
===> 23.3.2 satisfies the requirement for minimum OTP version 21
===> Setting paths to [deps]
===> Compile (apps)
===> Running hooks for compile in app beamoji (/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji) with configuration:
===>   {pre_hooks, []}.
===> run_hooks("/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji", pre_hooks, compile) -> no hooks defined

===> Running hooks for erlc_compile in app beamoji (/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji) with configuration:
===>   {pre_hooks, []}.
===> run_hooks("/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji", pre_hooks, erlc_compile) -> no hooks defined

===> Setting paths to [deps]
===> Analyzing applications...
===> Compiling beamoji
===> compile options: {erl_opts, [warn_unused_vars,warn_export_all,
                                         warn_shadow_vars,warn_unused_import,
                                         warn_unused_function,warn_bif_clash,
                                         warn_unused_record,
                                         warn_deprecated_function,
                                         warn_obsolete_guard,
                                         strict_validation,warn_export_vars,
                                         warn_exported_vars,debug_info]}.
===> files to analyze ["/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji/src/beamoji_prv.erl",
                              "/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji/src/beamoji.erl"]
===> Starting 1 worker(s)
===>      Compiled beamoji_prv.erl
===> Running hooks for erlc_compile in app beamoji (/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji) with configuration:
===>   {post_hooks, []}.
===> run_hooks("/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji", post_hooks, erlc_compile) -> no hooks defined

===> Running hooks for app_compile in app beamoji (/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji) with configuration:
===>   {pre_hooks, []}.
===> run_hooks("/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji", pre_hooks, app_compile) -> no hooks defined

===> Setting paths to [plugins]
===> Setting paths to [deps]
===> Running hooks for app_compile in app beamoji (/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji) with configuration:
===>   {post_hooks, []}.
===> run_hooks("/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji", post_hooks, app_compile) -> no hooks defined

===> Running hooks for compile in app beamoji (/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji) with configuration:
===>   {post_hooks, []}.
===> run_hooks("/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji", post_hooks, compile) -> no hooks defined

===> Setting paths to [plugins]
===> Setting paths to [plugins]
===> Expanded command sequence to be run: [help]
===> Running provider: help
Rebar3 is a tool for working with Erlang projects.
…

  Set the environment variable DEBUG=1 for detailed output.

Several tasks are available:
…
xref              Run cross reference analysis.
===> Uncaught error in rebar_core. Run with DIAGNOSTIC=1 to see stacktrace or consult rebar3.crashdump
===> Uncaught error: badarg
===> Stack trace to the error location:
[{io,format,
     [<0.64.0>,"~s~s~s~n",
      [[129668],
       [" "," "," "," "," "," "," "," "," "," "," "," "," "," "," "," "," "],
       [129668,32,8674,32,129327]]],
     []},
 {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
 {providers,namespace_help,2,
            [{file,"/home/runner/work/rebar3/rebar3/_build/default/lib/providers/src/providers.erl"},
             {line,317}]},
 {rebar_prv_help,help,1,
                 [{file,"/home/runner/work/rebar3/rebar3/src/rebar_prv_help.erl"},
                  {line,64}]},
 {rebar_prv_help,do,1,
                 [{file,"/home/runner/work/rebar3/rebar3/src/rebar_prv_help.erl"},
                  {line,39}]},
 {rebar_core,do,2,
             [{file,"/home/runner/work/rebar3/rebar3/src/rebar_core.erl"},
              {line,155}]},
 {rebar3,run_aux,2,
         [{file,"/home/runner/work/rebar3/rebar3/src/rebar3.erl"},{line,182}]},
 {rebar3,main,1,
         [{file,"/home/runner/work/rebar3/rebar3/src/rebar3.erl"},{line,66}]}]
===> When submitting a bug report, please include the output of `rebar3 report "your command"`

If we change the name of the provider to emojify and remove the emojis from the short desc, it works. Even more, emojis are fine in the description field…

$ rebar3 help emojify
===> Analyzing applications...
===> Compiling beamoji
🪄📜 ⇢ 📃✨ ⇢ 🤯
Usage: rebar3 emojify

But not in rebar3 warnings…

$ DIAGNOSTIC=1 rebar3 emojify
===> Load global config file /Users/fernandobenavides/.config/rebar3/rebar.config
===> 23.3.2 satisfies the requirement for minimum OTP version 21
===> Setting paths to [deps]
===> Compile (apps)
===> Setting paths to [plugins]
===> Setting paths to [deps]
===> Setting paths to [plugins]
===> Setting paths to [plugins]
===> 23.3.2 satisfies the requirement for minimum OTP version 21
===> 23.3.2 satisfies the requirement for minimum OTP version 21
===> Setting paths to [deps]
===> Compile (apps)
===> Running hooks for compile in app beamoji (/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji) with configuration:
===>   {pre_hooks, []}.
===> run_hooks("/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji", pre_hooks, compile) -> no hooks defined

===> Running hooks for erlc_compile in app beamoji (/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji) with configuration:
===>   {pre_hooks, []}.
===> run_hooks("/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji", pre_hooks, erlc_compile) -> no hooks defined

===> Setting paths to [deps]
===> Analyzing applications...
===> Compiling beamoji
===> compile options: {erl_opts, [warn_unused_vars,warn_export_all,
                                         warn_shadow_vars,warn_unused_import,
                                         warn_unused_function,warn_bif_clash,
                                         warn_unused_record,
                                         warn_deprecated_function,
                                         warn_obsolete_guard,
                                         strict_validation,warn_export_vars,
                                         warn_exported_vars,debug_info]}.
===> files to analyze ["/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji/src/beamoji_prv.erl",
                              "/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji/src/beamoji.erl"]
===> Starting 0 worker(s)
===> Running hooks for erlc_compile in app beamoji (/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji) with configuration:
===>   {post_hooks, []}.
===> run_hooks("/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji", post_hooks, erlc_compile) -> no hooks defined

===> Running hooks for app_compile in app beamoji (/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji) with configuration:
===>   {pre_hooks, []}.
===> run_hooks("/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji", pre_hooks, app_compile) -> no hooks defined

===> Setting paths to [plugins]
===> Setting paths to [deps]
===> Running hooks for app_compile in app beamoji (/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji) with configuration:
===>   {post_hooks, []}.
===> run_hooks("/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji", post_hooks, app_compile) -> no hooks defined

===> Running hooks for compile in app beamoji (/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji) with configuration:
===>   {post_hooks, []}.
===> run_hooks("/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji", post_hooks, compile) -> no hooks defined

===> Setting paths to [plugins]
===> Setting paths to [plugins]
===> Expanded command sequence to be run: [emojify]
===> Running provider: emojify
===> Uncaught error in rebar_core. Run with DIAGNOSTIC=1 to see stacktrace or consult rebar3.crashdump
===> Uncaught error: badarg
===> Stack trace to the error location:
[{io_lib,format,
         ["\e[0;35m~s~s\e[0m",
          ["===> ",[129668,32,99,97,108,108,101,100,32,9888,65039,126,110]]],
         [{file,"io_lib.erl"},{line,187}]},
 {ec_cmd_log,colorize,4,
             [{file,"/home/runner/work/rebar3/rebar3/_build/default/lib/erlware_commons/src/ec_cmd_log.erl"},
              {line,257}]},
 {ec_cmd_log,warn,3,
             [{file,"/home/runner/work/rebar3/rebar3/_build/default/lib/erlware_commons/src/ec_cmd_log.erl"},
              {line,184}]},
 {beamoji_prv,do,1,
              [{file,"/Users/fernandobenavides/Projects/spawnfest/beamoji_test/_checkouts/beamoji/src/beamoji_prv.erl"},
               {line,24}]},
 {rebar_core,do,2,
             [{file,"/home/runner/work/rebar3/rebar3/src/rebar_core.erl"},
              {line,155}]},
 {rebar3,run_aux,2,
         [{file,"/home/runner/work/rebar3/rebar3/src/rebar3.erl"},{line,182}]},
 {rebar3,main,1,
         [{file,"/home/runner/work/rebar3/rebar3/src/rebar3.erl"},{line,66}]},
 {escript,run,2,[{file,"escript.erl"},{line,758}]}]
===> When submitting a bug report, please include the output of `rebar3 report "your command"`

Expected behaviour

I would expect to be able to do the following…

$ rebar3 🪄
===> Analyzing applications...
===> Compiling beamoji
===> 🪄 called ⚠️

[ferd]

To support the name, you probably need to go patch:

• https:/tsloughter/providers/blob/master/src/providers.erl#L313-L327 to use the ~t flag (for the help)
• https:/erlang/rebar3/blob/master/src/rebar_core.erl to do the same (debug printing)

For the warning, I'm digging in:

• https:/erlware/erlware_commons/blob/master/src/ec_cmd_log.erl#L257 and this seems to already be supported
• I figure the actual failing call is in https:/project-fifo/cf/blob/master/src/cf.erl#L55-L56 which fails with a deep list passed despite previously flattening it in https:/project-fifo/cf/blob/master/src/cf.erl#L118-L119

This would be worth digging into more, though I see this is a spawnfest thing for you, so I figure for the time being:

a) this won't be fixed and released on time for the contest
b) I'm also participating and that makes it even harder

I figure you have a workaround by using non-colorized printed text or by passing the calls tunneled via rebar_api ?

[elbrujohalcon]

What are you doing here, @ferd ???
Go back to your @spawnfest project, man!