Hoa\Compiler 3.16.08.15

announcements
1

Hello all :slight_smile:,

I am very exciting to announce that Hoa\Compiler 3.16.08.15 has been released, aka the performance branch.

Short changelog

Lexer:

  • Acts like an iterator, which brings constant memory usage while lexing small, medium or huge data,

Parser and rules:

  • Reduce method calls, thus indirections,
  • Reduce memory with transitional rules,
  • Introduce the Hoa\Compiler\Llk::save method to transform an in-memory parser to PHP code (when running, it builds the same parser, state is reset).

Grammar:

  • Introduce pragmas with the %pragma instructions, two already exist:
    • lexer.unicode to enable or disable Unicode support in the lexer,
  • parser.lookahead to set the value of k in LL(k), i.e. the number of tokens to look ahead.

Hoa\Json uses the lexer.unicode pragmas to fully implementing the RFC7159, especially for the UTF-8 implementation with surrogate pairs format. This library also uses the parser.lookahead pragma to set k to 0 (in LL(k)), so we have a LL(0) parser.

Quality:

  • New unit test suites have been written,
  • New integration test suites have been written,
  • Existing integration test suites have been enhanced,
  • 2 bugs have been found and fixed,
  • Numbers:
    • 24 test suites,
    • 136 test cases,
    • 320,242 assertions.

Performance?

Yup. Much better. The blog post will detail the numbers but we observe:

  • 43-48% less memory in the lexer, and
  • 5% faster with the parser compiled to PHP code instead of serialized.

Next?

  1. Publish the blog post,
  2. Write a Hoa\Slice library to avoid copies of big arrays in the compiler. We must discuss about that.