Colour formatting gets confused due in specific situations with comment lines and another string containing lots of commas

[adam-jones-net]

Prerequisites

• Put an X between the brackets on this line if you have done all of the following:
  • Reproduced the problem in Safe Mode: https://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
  • Followed all applicable steps in the debugging guide: https://flight-manual.atom.io/hacking-atom/sections/debugging/
  • Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
  • Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
  • Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description

Colour formatting gets confused due in specific situations with comment lines and another string containing lots of commas.

Steps to Reproduce

Create a new PHP file in atom and then paste the following code into it.

<?php
//
$query='SELECT test1, test2, test3,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,';
// the colour formatting should be broken already by the time of this comment line
?>

Expected behavior:

Normal colouring of syntax

Actual behavior:

The colour syntax is wrong. Delete either the top "//" or some of the content in the $query variable and it resolves itself.

Reproduces how often:

All the time

Versions

Atom : 1.58.0
Electron: 9.4.4
Chrome : 83.0.4103.122
Node : 12.14.1
Windows 10 Build 21H1

[KapitanOczywisty]

@Arkid This is caused by token limit per one line, which is equal to 100 by default. Each coma is treated as separate token.

If you need this limit increased you can add Init script

// init.js
let php = atom.grammars.grammarForScopeName('text.html.php');
// tokens per line (default: 100)
php.maxTokensPerLine = 200;
// tokenize until N characters per line (default: 1000)
// php.maxLineLength = 2000;

And coffee variant

# init.coffee
php = atom.grammars.grammarForScopeName('text.html.php')
php.maxTokensPerLine = 200
# php.maxLineLength = 2000