Parse lists in LaTeX macros

[notjagan]

At the moment, we have macros that use lists of elements surrounded by parentheses and delimited by semicolons/commas. These elements are sometimes not covered by lit; for example, \cos\tha is not matched since it is two literals chained together.

As such, using lit in the auto-generated rule doesn't find occurrences of the macro when a case like this appears.

We attempted to create a new rule that accepted such lists and added it as an option to lit but the return types were radically different and the program was unable to handle the lists properly.

As of now, we aren't sure of how we should approach a fix to this problem.

@physikerwelt @HowardCohl

[notjagan]

We think that this may be possible by adding a new rule higher up than lit but we are unsure at the moment. Additionally, we don't know where in the hierarchy it would stand.

[physikerwelt]

I don't understand the problem. What is the definition of matched and converted?

[notjagan]

Here's an example of the problem.

p_n(\cos\tha;a,b,c,d\,|\, q) (on line 1261 of KLSadd.tex)

It should match p_n followed by 6 arguments - in other words, we generate a rule in pegjs that looks for p_n followed by 6 lits encased in parentheses and delimited by commas. This rule is generated based on the following macro definition:

\defSpecFun{AskeyWilson}[1]{p_{#1}}[meaning=Askey-Wilson-polynomial-p]{6}[\!\left(#1;#2,#3,#4,#5\,|\,#6\right)]

However, in the previous example, \cos\tha is not a lit. This is because it is actually two lits chained together. As such, it isn't matched by the automatically generated pegjs rule, and we're not sure how to make the rule match such cases.

[physikerwelt]

in other words, we generate a rule in pegjs that looks for p_n followed by 6 lits encased in parentheses
I think this are not only other words but also other things.

The link https:/notjagan/texer/blob/master/lib/DRMFfcns.sty#L74 does not work for me;-(

[HowardCohl]

@physikerwelt They mistakenly put the sty file on github (it is private), they have removed it. Look at the re-edited post.

[notjagan]

I had to remove the file because it turned out to be private. However, the relevant line is now in my above comment.

[notjagan]

That was unintentional; I apologize.

[physikerwelt]

ok, I finally understand the problem. But I still don't know the solution.

[physikerwelt]

How about using something more general than lit such as expr?

[physikerwelt]

Maybe the additional element introduced by @ClaudeZou DRMF/texvcjs@2049d82 are also helpful here?

[poortho]

We tried that previously. Unfortunately, it didn't seem to work. It's possible that we implemented it wrong, but I don't think they're the same type of issue.

[HowardCohl]

@physikerwelt can you try to explain why you think it should work, or perhaps a suggested modification which would allow it to work?

[poortho]

@physikerwelt Dr. Cohl mentioned that you had a potential solution to our problem; could you explain what it is/how it works?

[physikerwelt]

@ClaudeZou I got the impression that your current approach is an uphill battle. Instead of improving the texvc rules, I propose to build another level of abstraction on top of texvc.

texvc

texvc is supposed to parse latex and to parse it using the same logic as the tex parser uses. This deals about understanding the structure of TeX impression with regard to the tex syntax. More specifically frac is a function with two arguments in the TeX language. However, f(a,b) is neither a function nor any other object with structure from a TeX point of view. It’s just a list of symbols. However, DRMF specific extensions of TeX/LaTeX such as the @ symbols are supposed to end up in the texvc grammar.

texer

Now, there is texer. Like texvcinfo which extracts identifiers from the texvc token stream, texer is supposed discover possible replacements of regular latex symbols with semantic latex macros.

texvcinfo

I think to get a better understanding of how that could be done you should have a look at the new texvcinfo ast visitors that extract identifiers.

[physikerwelt]

@HowardCohl @AndreG-P can you check if https:/physikerwelt/texvcinfo/blob/master/README.md#implementation-details-on-identifier-extraction helps to understand how the identifier extraction in texvcinfo works?