messages.json gets deleted when there is a newline in translation string

[mcku]

Hi,

I find js-lingui very useful and like the approach. Due to some other circumstances I have to use typescript and babel 7. Therefore using local packages of your repo in my project. Currently I am using simple <Trans>{children}</Trans> for translating content to Arabic. The initial extraction is fine and I get

Messages extracted!

Catalog statistics:

┌──────────┬─────────────┬─────────┐
│ Language │ Total count │ Missing │
├──────────┼─────────────┼─────────┤
│ ar       │      5      │    5    │
│ en       │      5      │    5    │
└──────────┴─────────────┴─────────┘

And then I fill in the ar part. Then when I run the lingui extract command again, I am observing that the message ids become the langugage ids. and the whole locale/ar/messages.json content consiting of 'undefined' line only. And the output of lingui extract becomes:

Catalog statistics:
┌────────────┬─────────────┬─────────┐
│ Language   │ Total count │ Missing │
├────────────┼─────────────┼─────────┤
│ msg.header │      2      │    2    │
│ aboutus.p1 │      2      │    2    │
│ aboutus.p2 │      2      │    2    │
│ aboutus.p3 │      2      │    2    │
│ aboutus.p4 │      2      │    2    │
│ en         │      5      │    5    │
└────────────┴─────────────┴─────────┘

I have copied the messages.json to somewhere and cleaned up the locale directory (rm -rf locale). Then generated it again and copied the messages.json file manually and re-run the extract and the same thing can be reproduced.

Then I have noticed that the messages.json line contains multi line strings (i.e. newline in quoted string). It may be good to have file syntax checking on messages before/while lingui runs extraction.
And it may be good not to delete the messages.json file (by leaving it with undefined word only), in the mentioned case.

Fixed the multi lines and everything is working fine again. (But in some cases this somewhat might be a catastrope to some.) Thanks, and keep up the good work!

[tricoder42]

Hello @mcku,
thank you for your kind words! I'm glad you like this project 🎉

Therefore using local packages of your repo in my project.

So, you forked the project and added support for typescript and babel 7? How did it went, was it difficult? There's some work on Typescript plugin by @johansigfrids (#48), but I didn't have time to dig into it as I don't have any experience with typescript.

Then I have noticed that the messages.json line contains multi line strings (i.e. newline in quoted string). It may be good to have file syntax checking on messages before/while lingui runs extraction.

I'm gonna try it, but I was fixing babel-plugin-lingui-transform-react so it always remove newlines. Do you use translations in vanilla JS? That's the only plugin which leaves newlines in messages.

There's also basic syntax checking, because message format is generated by script and should be valid. Plugins are checking for missing params, invalid values, plural forms, etc.

And it may be good not to delete the messages.json file (by leaving it with undefined word only), in the mentioned case.

Actually, when you plugin finds an error, it throws SyntaxError exception and the compilation is over.

Anyway, thank you for your inputs! I would like to know more about your use case.

[mcku]

So, you forked the project and added support for typescript and babel 7? How did it went, was it difficult? There's some work on Typescript plugin by @johansigfrids (#48), but I didn't have time to dig into it as I don't have any experience with typescript.

Yes I did fork it and noticed you guys are using lerna and tried it but my fork is not current, in the end I have also added yarn workspaces in order to do things locally. What i did was just to replace babel dependencies with the babel 7 versions. Also I could not use the babel preset because the new babel does not want objects to be exported, but only functions. Maybe you can fix that when migrating to babel7. I tried to do it but did not work. A workaround was to use the plugins directly. I think it was a lucky situation, it all worked.

Also have seen the typescript plugin. Essentially, what TS does is to generate javascript using tsc. In that sense, in any typescript project there is already some configuration that converts ts/tsx into js/jsx. Therefore running tsc is fine. The plugin may be too restrictive, because my tsconfig.json outputs the whole project into a subtree already as jsx files, where <Trans> tags can be parsed by js-lingui as usual.

Anyway, thank you for your inputs! I would like to know more about your use case.

Yarn workspace mechanism handles local symlinking as long as local versions are referenced within package.json. Therefore bumped all versions into a same level to remember each package's version to easily make it appear consistent.

Project root package.json:

"private":true,
"workspaces":["packages/*"]

and in packages/ also replaced all references to babel 6 with babel 7 in within these packages, and replaced all references to external versions of lingui packages with the new hypothetical version.

babel-preset-lingui-js/	   lingui-i18n/       babel-preset-lingui-react/  lingui-loader/
babel-plugin-lingui-extract-messages/  lingui-cli/		   lingui-react/
babel-plugin-lingui-transform-js/      lingui-conf/		   typescript-lingui-extract-messages/
babel-plugin-lingui-transform-react/   lingui-formats/

I'm gonna try it, but I was fixing babel-plugin-lingui-transform-react so it always remove newlines. Do you use translations in vanilla JS? That's the only plugin which leaves newlines in messages.

No, but I am using typescript and babel loaders together such that, ts outputs ES6, and babel takes it from there. I found this flexible and fast enough. For instance:

rules: [
      {
        test: /\.tsx?$/,
        use: [{
          loader: 'babel-loader'
        },{
          loader: 'awesome-typescript-loader'
        }],
        exclude: [/node_modules/]
      },

So in tsconfig.json just point to an output directory (compilerOptions outDir), and in lingui (via package.json), use that directory as srcPathDirs. Then run manually tsc and lingui extract, lingui compile by hand. That would be nice if there was a webpack plugin for extraction and compilation as a declarative plugin, too.

I am happy to share input and should more input or tests be necessary, no problem at all. This plugin is very useful and thanks for developing it.

[tricoder42]

Hey @mcku, sorry for delayed response!

New major version has just been released. Project uses yarn workspaces now and plugins are compatible with babel 7.

I never had problem with multiline strings. There're even test cases which check that newlines are handled correctly. Could you please post an example of <Trans> tag which produces faulty message?

[mcku]

@tricoder42 Thanks a lot!! I am closing this then.