diff options
Diffstat (limited to 'plugins/Pcre16/docs')
120 files changed, 0 insertions, 54362 deletions
diff --git a/plugins/Pcre16/docs/AUTHORS b/plugins/Pcre16/docs/AUTHORS deleted file mode 100644 index 5eee1af4c6..0000000000 --- a/plugins/Pcre16/docs/AUTHORS +++ /dev/null @@ -1,45 +0,0 @@ -THE MAIN PCRE LIBRARY ---------------------- - -Written by: Philip Hazel -Email local part: ph10 -Email domain: cam.ac.uk - -University of Cambridge Computing Service, -Cambridge, England. - -Copyright (c) 1997-2014 University of Cambridge -All rights reserved - - -PCRE JUST-IN-TIME COMPILATION SUPPORT -------------------------------------- - -Written by: Zoltan Herczeg -Email local part: hzmester -Emain domain: freemail.hu - -Copyright(c) 2010-2014 Zoltan Herczeg -All rights reserved. - - -STACK-LESS JUST-IN-TIME COMPILER --------------------------------- - -Written by: Zoltan Herczeg -Email local part: hzmester -Emain domain: freemail.hu - -Copyright(c) 2009-2014 Zoltan Herczeg -All rights reserved. - - -THE C++ WRAPPER LIBRARY ------------------------ - -Written by: Google Inc. - -Copyright (c) 2007-2012 Google Inc -All rights reserved - -#### diff --git a/plugins/Pcre16/docs/COPYING b/plugins/Pcre16/docs/COPYING deleted file mode 100644 index 58eed01b61..0000000000 --- a/plugins/Pcre16/docs/COPYING +++ /dev/null @@ -1,5 +0,0 @@ -PCRE LICENCE - -Please see the file LICENCE in the PCRE distribution for licensing details. - -End diff --git a/plugins/Pcre16/docs/ChangeLog b/plugins/Pcre16/docs/ChangeLog deleted file mode 100644 index 7801ef8411..0000000000 --- a/plugins/Pcre16/docs/ChangeLog +++ /dev/null @@ -1,5477 +0,0 @@ -ChangeLog for PCRE ------------------- - -Version 8.35 04-April-2014 --------------------------- - -1. A new flag is set, when property checks are present in an XCLASS. - When this flag is not set, PCRE can perform certain optimizations - such as studying these XCLASS-es. - -2. The auto-possessification of character sets were improved: a normal - and an extended character set can be compared now. Furthermore - the JIT compiler optimizes more character set checks. - -3. Got rid of some compiler warnings for potentially uninitialized variables - that show up only when compiled with -O2. - -4. A pattern such as (?=ab\K) that uses \K in an assertion can set the start - of a match later then the end of the match. The pcretest program was not - handling the case sensibly - it was outputting from the start to the next - binary zero. It now reports this situation in a message, and outputs the - text from the end to the start. - -5. Fast forward search is improved in JIT. Instead of the first three - characters, any three characters with fixed position can be searched. - Search order: first, last, middle. - -6. Improve character range checks in JIT. Characters are read by an inprecise - function now, which returns with an unknown value if the character code is - above a certain treshold (e.g: 256). The only limitation is that the value - must be bigger than the treshold as well. This function is useful, when - the characters above the treshold are handled in the same way. - -7. The macros whose names start with RAWUCHAR are placeholders for a future - mode in which only the bottom 21 bits of 32-bit data items are used. To - make this more memorable for those maintaining the code, the names have - been changed to start with UCHAR21, and an extensive comment has been added - to their definition. - -8. Add missing (new) files sljitNativeTILEGX.c and sljitNativeTILEGX-encoder.c - to the export list in Makefile.am (they were accidentally omitted from the - 8.34 tarball). - -9. The informational output from pcretest used the phrase "starting byte set" - which is inappropriate for the 16-bit and 32-bit libraries. As the output - for "first char" and "need char" really means "non-UTF-char", I've changed - "byte" to "char", and slightly reworded the output. The documentation about - these values has also been (I hope) clarified. - -10. Another JIT related optimization: use table jumps for selecting the correct - backtracking path, when more than four alternatives are present inside a - bracket. - -11. Empty match is not possible, when the minimum length is greater than zero, - and there is no \K in the pattern. JIT should avoid empty match checks in - such cases. - -12. In a caseless character class with UCP support, when a character with more - than one alternative case was not the first character of a range, not all - the alternative cases were added to the class. For example, s and \x{17f} - are both alternative cases for S: the class [RST] was handled correctly, - but [R-T] was not. - -13. The configure.ac file always checked for pthread support when JIT was - enabled. This is not used in Windows, so I have put this test inside a - check for the presence of windows.h (which was already tested for). - -14. Improve pattern prefix search by a simplified Boyer-Moore algorithm in JIT. - The algorithm provides a way to skip certain starting offsets, and usually - faster than linear prefix searches. - -15. Change 13 for 8.20 updated RunTest to check for the 'fr' locale as well - as for 'fr_FR' and 'french'. For some reason, however, it then used the - Windows-specific input and output files, which have 'french' screwed in. - So this could never have worked. One of the problems with locales is that - they aren't always the same. I have now updated RunTest so that it checks - the output of the locale test (test 3) against three different output - files, and it allows the test to pass if any one of them matches. With luck - this should make the test pass on some versions of Solaris where it was - failing. Because of the uncertainty, the script did not used to stop if - test 3 failed; it now does. If further versions of a French locale ever - come to light, they can now easily be added. - -16. If --with-pcregrep-bufsize was given a non-integer value such as "50K", - there was a message during ./configure, but it did not stop. This now - provokes an error. The invalid example in README has been corrected. - If a value less than the minimum is given, the minimum value has always - been used, but now a warning is given. - -17. If --enable-bsr-anycrlf was set, the special 16/32-bit test failed. This - was a bug in the test system, which is now fixed. Also, the list of various - configurations that are tested for each release did not have one with both - 16/32 bits and --enable-bar-anycrlf. It now does. - -18. pcretest was missing "-C bsr" for displaying the \R default setting. - -19. Little endian PowerPC systems are supported now by the JIT compiler. - -20. The fast forward newline mechanism could enter to an infinite loop on - certain invalid UTF-8 input. Although we don't support these cases - this issue can be fixed by a performance optimization. - -21. Change 33 of 8.34 is not sufficient to ensure stack safety because it does - not take account if existing stack usage. There is now a new global - variable called pcre_stack_guard that can be set to point to an external - function to check stack availability. It is called at the start of - processing every parenthesized group. - -22. A typo in the code meant that in ungreedy mode the max/min qualifier - behaved like a min-possessive qualifier, and, for example, /a{1,3}b/U did - not match "ab". - -23. When UTF was disabled, the JIT program reported some incorrect compile - errors. These messages are silenced now. - -24. Experimental support for ARM-64 and MIPS-64 has been added to the JIT - compiler. - -25. Change all the temporary files used in RunGrepTest to be different to those - used by RunTest so that the tests can be run simultaneously, for example by - "make -j check". - - -Version 8.34 15-December-2013 ------------------------------ - -1. Add pcre[16|32]_jit_free_unused_memory to forcibly free unused JIT - executable memory. Patch inspired by Carsten Klein. - -2. ./configure --enable-coverage defined SUPPORT_GCOV in config.h, although - this macro is never tested and has no effect, because the work to support - coverage involves only compiling and linking options and special targets in - the Makefile. The comment in config.h implied that defining the macro would - enable coverage support, which is totally false. There was also support for - setting this macro in the CMake files (my fault, I just copied it from - configure). SUPPORT_GCOV has now been removed. - -3. Make a small performance improvement in strlen16() and strlen32() in - pcretest. - -4. Change 36 for 8.33 left some unreachable statements in pcre_exec.c, - detected by the Solaris compiler (gcc doesn't seem to be able to diagnose - these cases). There was also one in pcretest.c. - -5. Cleaned up a "may be uninitialized" compiler warning in pcre_exec.c. - -6. In UTF mode, the code for checking whether a group could match an empty - string (which is used for indefinitely repeated groups to allow for - breaking an infinite loop) was broken when the group contained a repeated - negated single-character class with a character that occupied more than one - data item and had a minimum repetition of zero (for example, [^\x{100}]* in - UTF-8 mode). The effect was undefined: the group might or might not be - deemed as matching an empty string, or the program might have crashed. - -7. The code for checking whether a group could match an empty string was not - recognizing that \h, \H, \v, \V, and \R must match a character. - -8. Implemented PCRE_INFO_MATCH_EMPTY, which yields 1 if the pattern can match - an empty string. If it can, pcretest shows this in its information output. - -9. Fixed two related bugs that applied to Unicode extended grapheme clusters - that were repeated with a maximizing qualifier (e.g. \X* or \X{2,5}) when - matched by pcre_exec() without using JIT: - - (a) If the rest of the pattern did not match after a maximal run of - grapheme clusters, the code for backing up to try with fewer of them - did not always back up over a full grapheme when characters that do not - have the modifier quality were involved, e.g. Hangul syllables. - - (b) If the match point in a subject started with modifier character, and - there was no match, the code could incorrectly back up beyond the match - point, and potentially beyond the first character in the subject, - leading to a segfault or an incorrect match result. - -10. A conditional group with an assertion condition could lead to PCRE - recording an incorrect first data item for a match if no other first data - item was recorded. For example, the pattern (?(?=ab)ab) recorded "a" as a - first data item, and therefore matched "ca" after "c" instead of at the - start. - -11. Change 40 for 8.33 (allowing pcregrep to find empty strings) showed up a - bug that caused the command "echo a | ./pcregrep -M '|a'" to loop. - -12. The source of pcregrep now includes z/OS-specific code so that it can be - compiled for z/OS as part of the special z/OS distribution. - -13. Added the -T and -TM options to pcretest. - -14. The code in pcre_compile.c for creating the table of named capturing groups - has been refactored. Instead of creating the table dynamically during the - actual compiling pass, the information is remembered during the pre-compile - pass (on the stack unless there are more than 20 named groups, in which - case malloc() is used) and the whole table is created before the actual - compile happens. This has simplified the code (it is now nearly 150 lines - shorter) and prepared the way for better handling of references to groups - with duplicate names. - -15. A back reference to a named subpattern when there is more than one of the - same name now checks them in the order in which they appear in the pattern. - The first one that is set is used for the reference. Previously only the - first one was inspected. This change makes PCRE more compatible with Perl. - -16. Unicode character properties were updated from Unicode 6.3.0. - -17. The compile-time code for auto-possessification has been refactored, based - on a patch by Zoltan Herczeg. It now happens after instead of during - compilation. The code is cleaner, and more cases are handled. The option - PCRE_NO_AUTO_POSSESS is added for testing purposes, and the -O and /O - options in pcretest are provided to set it. It can also be set by - (*NO_AUTO_POSSESS) at the start of a pattern. - -18. The character VT has been added to the default ("C" locale) set of - characters that match \s and are generally treated as white space, - following this same change in Perl 5.18. There is now no difference between - "Perl space" and "POSIX space". Whether VT is treated as white space in - other locales depends on the locale. - -19. The code for checking named groups as conditions, either for being set or - for being recursed, has been refactored (this is related to 14 and 15 - above). Processing unduplicated named groups should now be as fast at - numerical groups, and processing duplicated groups should be faster than - before. - -20. Two patches to the CMake build system, by Alexander Barkov: - - (1) Replace the "source" command by "." in CMakeLists.txt because - "source" is a bash-ism. - - (2) Add missing HAVE_STDINT_H and HAVE_INTTYPES_H to config-cmake.h.in; - without these the CMake build does not work on Solaris. - -21. Perl has changed its handling of \8 and \9. If there is no previously - encountered capturing group of those numbers, they are treated as the - literal characters 8 and 9 instead of a binary zero followed by the - literals. PCRE now does the same. - -22. Following Perl, added \o{} to specify codepoints in octal, making it - possible to specify values greater than 0777 and also making them - unambiguous. - -23. Perl now gives an error for missing closing braces after \x{... instead of - treating the string as literal. PCRE now does the same. - -24. RunTest used to grumble if an inappropriate test was selected explicitly, - but just skip it when running all tests. This make it awkward to run ranges - of tests when one of them was inappropriate. Now it just skips any - inappropriate tests, as it always did when running all tests. - -25. If PCRE_AUTO_CALLOUT and PCRE_UCP were set for a pattern that contained - character types such as \d or \w, too many callouts were inserted, and the - data that they returned was rubbish. - -26. In UCP mode, \s was not matching two of the characters that Perl matches, - namely NEL (U+0085) and MONGOLIAN VOWEL SEPARATOR (U+180E), though they - were matched by \h. The code has now been refactored so that the lists of - the horizontal and vertical whitespace characters used for \h and \v (which - are defined only in one place) are now also used for \s. - -27. Add JIT support for the 64 bit TileGX architecture. - Patch by Jiong Wang (Tilera Corporation). - -28. Possessive quantifiers for classes (both explicit and automatically - generated) now use special opcodes instead of wrapping in ONCE brackets. - -29. Whereas an item such as A{4}+ ignored the possessivenes of the quantifier - (because it's meaningless), this was not happening when PCRE_CASELESS was - set. Not wrong, but inefficient. - -30. Updated perltest.pl to add /u (force Unicode mode) when /W (use Unicode - properties for \w, \d, etc) is present in a test regex. Otherwise if the - test contains no characters greater than 255, Perl doesn't realise it - should be using Unicode semantics. - -31. Upgraded the handling of the POSIX classes [:graph:], [:print:], and - [:punct:] when PCRE_UCP is set so as to include the same characters as Perl - does in Unicode mode. - -32. Added the "forbid" facility to pcretest so that putting tests into the - wrong test files can sometimes be quickly detected. - -33. There is now a limit (default 250) on the depth of nesting of parentheses. - This limit is imposed to control the amount of system stack used at compile - time. It can be changed at build time by --with-parens-nest-limit=xxx or - the equivalent in CMake. - -34. Character classes such as [A-\d] or [a-[:digit:]] now cause compile-time - errors. Perl warns for these when in warning mode, but PCRE has no facility - for giving warnings. - -35. Change 34 for 8.13 allowed quantifiers on assertions, because Perl does. - However, this was not working for (?!) because it is optimized to (*FAIL), - for which PCRE does not allow quantifiers. The optimization is now disabled - when a quantifier follows (?!). I can't see any use for this, but it makes - things uniform. - -36. Perl no longer allows group names to start with digits, so I have made this - change also in PCRE. It simplifies the code a bit. - -37. In extended mode, Perl ignores spaces before a + that indicates a - possessive quantifier. PCRE allowed a space before the quantifier, but not - before the possessive +. It now does. - -38. The use of \K (reset reported match start) within a repeated possessive - group such as (a\Kb)*+ was not working. - -40. Document that the same character tables must be used at compile time and - run time, and that the facility to pass tables to pcre_exec() and - pcre_dfa_exec() is for use only with saved/restored patterns. - -41. Applied Jeff Trawick's patch CMakeLists.txt, which "provides two new - features for Builds with MSVC: - - 1. Support pcre.rc and/or pcreposix.rc (as is already done for MinGW - builds). The .rc files can be used to set FileDescription and many other - attributes. - - 2. Add an option (-DINSTALL_MSVC_PDB) to enable installation of .pdb files. - This allows higher-level build scripts which want .pdb files to avoid - hard-coding the exact files needed." - -42. Added support for [[:<:]] and [[:>:]] as used in the BSD POSIX library to - mean "start of word" and "end of word", respectively, as a transition aid. - -43. A minimizing repeat of a class containing codepoints greater than 255 in - non-UTF 16-bit or 32-bit modes caused an internal error when PCRE was - compiled to use the heap for recursion. - -44. Got rid of some compiler warnings for unused variables when UTF but not UCP - is configured. - - -Version 8.33 28-May-2013 ------------------------- - -1. Added 'U' to some constants that are compared to unsigned integers, to - avoid compiler signed/unsigned warnings. Added (int) casts to unsigned - variables that are added to signed variables, to ensure the result is - signed and can be negated. - -2. Applied patch by Daniel Richard G for quashing MSVC warnings to the - CMake config files. - -3. Revise the creation of config.h.generic so that all boolean macros are - #undefined, whereas non-boolean macros are #ifndef/#endif-ed. This makes - overriding via -D on the command line possible. - -4. Changing the definition of the variable "op" in pcre_exec.c from pcre_uchar - to unsigned int is reported to make a quite noticeable speed difference in - a specific Windows environment. Testing on Linux did also appear to show - some benefit (and it is clearly not harmful). Also fixed the definition of - Xop which should be unsigned. - -5. Related to (4), changing the definition of the intermediate variable cc - in repeated character loops from pcre_uchar to pcre_uint32 also gave speed - improvements. - -6. Fix forward search in JIT when link size is 3 or greater. Also removed some - unnecessary spaces. - -7. Adjust autogen.sh and configure.ac to lose warnings given by automake 1.12 - and later. - -8. Fix two buffer over read issues in 16 and 32 bit modes. Affects JIT only. - -9. Optimizing fast_forward_start_bits in JIT. - -10. Adding support for callouts in JIT, and fixing some issues revealed - during this work. Namely: - - (a) Unoptimized capturing brackets incorrectly reset on backtrack. - - (b) Minimum length was not checked before the matching is started. - -11. The value of capture_last that is passed to callouts was incorrect in some - cases when there was a capture on one path that was subsequently abandoned - after a backtrack. Also, the capture_last value is now reset after a - recursion, since all captures are also reset in this case. - -12. The interpreter no longer returns the "too many substrings" error in the - case when an overflowing capture is in a branch that is subsequently - abandoned after a backtrack. - -13. In the pathological case when an offset vector of size 2 is used, pcretest - now prints out the matched string after a yield of 0 or 1. - -14. Inlining subpatterns in recursions, when certain conditions are fulfilled. - Only supported by the JIT compiler at the moment. - -15. JIT compiler now supports 32 bit Macs thanks to Lawrence Velazquez. - -16. Partial matches now set offsets[2] to the "bumpalong" value, that is, the - offset of the starting point of the matching process, provided the offsets - vector is large enough. - -17. The \A escape now records a lookbehind value of 1, though its execution - does not actually inspect the previous character. This is to ensure that, - in partial multi-segment matching, at least one character from the old - segment is retained when a new segment is processed. Otherwise, if there - are no lookbehinds in the pattern, \A might match incorrectly at the start - of a new segment. - -18. Added some #ifdef __VMS code into pcretest.c to help VMS implementations. - -19. Redefined some pcre_uchar variables in pcre_exec.c as pcre_uint32; this - gives some modest performance improvement in 8-bit mode. - -20. Added the PCRE-specific property \p{Xuc} for matching characters that can - be expressed in certain programming languages using Universal Character - Names. - -21. Unicode validation has been updated in the light of Unicode Corrigendum #9, - which points out that "non characters" are not "characters that may not - appear in Unicode strings" but rather "characters that are reserved for - internal use and have only local meaning". - -22. When a pattern was compiled with automatic callouts (PCRE_AUTO_CALLOUT) and - there was a conditional group that depended on an assertion, if the - assertion was false, the callout that immediately followed the alternation - in the condition was skipped when pcre_exec() was used for matching. - -23. Allow an explicit callout to be inserted before an assertion that is the - condition for a conditional group, for compatibility with automatic - callouts, which always insert a callout at this point. - -24. In 8.31, (*COMMIT) was confined to within a recursive subpattern. Perl also - confines (*SKIP) and (*PRUNE) in the same way, and this has now been done. - -25. (*PRUNE) is now supported by the JIT compiler. - -26. Fix infinite loop when /(?<=(*SKIP)ac)a/ is matched against aa. - -27. Fix the case where there are two or more SKIPs with arguments that may be - ignored. - -28. (*SKIP) is now supported by the JIT compiler. - -29. (*THEN) is now supported by the JIT compiler. - -30. Update RunTest with additional test selector options. - -31. The way PCRE handles backtracking verbs has been changed in two ways. - - (1) Previously, in something like (*COMMIT)(*SKIP), COMMIT would override - SKIP. Now, PCRE acts on whichever backtracking verb is reached first by - backtracking. In some cases this makes it more Perl-compatible, but Perl's - rather obscure rules do not always do the same thing. - - (2) Previously, backtracking verbs were confined within assertions. This is - no longer the case for positive assertions, except for (*ACCEPT). Again, - this sometimes improves Perl compatibility, and sometimes does not. - -32. A number of tests that were in test 2 because Perl did things differently - have been moved to test 1, because either Perl or PCRE has changed, and - these tests are now compatible. - -32. Backtracking control verbs are now handled in the same way in JIT and - interpreter. - -33. An opening parenthesis in a MARK/PRUNE/SKIP/THEN name in a pattern that - contained a forward subroutine reference caused a compile error. - -34. Auto-detect and optimize limited repetitions in JIT. - -35. Implement PCRE_NEVER_UTF to lock out the use of UTF, in particular, - blocking (*UTF) etc. - -36. In the interpreter, maximizing pattern repetitions for characters and - character types now use tail recursion, which reduces stack usage. - -37. The value of the max lookbehind was not correctly preserved if a compiled - and saved regex was reloaded on a host of different endianness. - -38. Implemented (*LIMIT_MATCH) and (*LIMIT_RECURSION). As part of the extension - of the compiled pattern block, expand the flags field from 16 to 32 bits - because it was almost full. - -39. Try madvise first before posix_madvise. - -40. Change 7 for PCRE 7.9 made it impossible for pcregrep to find empty lines - with a pattern such as ^$. It has taken 4 years for anybody to notice! The - original change locked out all matches of empty strings. This has been - changed so that one match of an empty string per line is recognized. - Subsequent searches on the same line (for colouring or for --only-matching, - for example) do not recognize empty strings. - -41. Applied a user patch to fix a number of spelling mistakes in comments. - -42. Data lines longer than 65536 caused pcretest to crash. - -43. Clarified the data type for length and startoffset arguments for pcre_exec - and pcre_dfa_exec in the function-specific man pages, where they were - explicitly stated to be in bytes, never having been updated. I also added - some clarification to the pcreapi man page. - -44. A call to pcre_dfa_exec() with an output vector size less than 2 caused - a segmentation fault. - - -Version 8.32 30-November-2012 ------------------------------ - -1. Improved JIT compiler optimizations for first character search and single - character iterators. - -2. Supporting IBM XL C compilers for PPC architectures in the JIT compiler. - Patch by Daniel Richard G. - -3. Single character iterator optimizations in the JIT compiler. - -4. Improved JIT compiler optimizations for character ranges. - -5. Rename the "leave" variable names to "quit" to improve WinCE compatibility. - Reported by Giuseppe D'Angelo. - -6. The PCRE_STARTLINE bit, indicating that a match can occur only at the start - of a line, was being set incorrectly in cases where .* appeared inside - atomic brackets at the start of a pattern, or where there was a subsequent - *PRUNE or *SKIP. - -7. Improved instruction cache flush for POWER/PowerPC. - Patch by Daniel Richard G. - -8. Fixed a number of issues in pcregrep, making it more compatible with GNU - grep: - - (a) There is now no limit to the number of patterns to be matched. - - (b) An error is given if a pattern is too long. - - (c) Multiple uses of --exclude, --exclude-dir, --include, and --include-dir - are now supported. - - (d) --exclude-from and --include-from (multiple use) have been added. - - (e) Exclusions and inclusions now apply to all files and directories, not - just to those obtained from scanning a directory recursively. - - (f) Multiple uses of -f and --file-list are now supported. - - (g) In a Windows environment, the default for -d has been changed from - "read" (the GNU grep default) to "skip", because otherwise the presence - of a directory in the file list provokes an error. - - (h) The documentation has been revised and clarified in places. - -9. Improve the matching speed of capturing brackets. - -10. Changed the meaning of \X so that it now matches a Unicode extended - grapheme cluster. - -11. Patch by Daniel Richard G to the autoconf files to add a macro for sorting - out POSIX threads when JIT support is configured. - -12. Added support for PCRE_STUDY_EXTRA_NEEDED. - -13. In the POSIX wrapper regcomp() function, setting re_nsub field in the preg - structure could go wrong in environments where size_t is not the same size - as int. - -14. Applied user-supplied patch to pcrecpp.cc to allow PCRE_NO_UTF8_CHECK to be - set. - -15. The EBCDIC support had decayed; later updates to the code had included - explicit references to (e.g.) \x0a instead of CHAR_LF. There has been a - general tidy up of EBCDIC-related issues, and the documentation was also - not quite right. There is now a test that can be run on ASCII systems to - check some of the EBCDIC-related things (but is it not a full test). - -16. The new PCRE_STUDY_EXTRA_NEEDED option is now used by pcregrep, resulting - in a small tidy to the code. - -17. Fix JIT tests when UTF is disabled and both 8 and 16 bit mode are enabled. - -18. If the --only-matching (-o) option in pcregrep is specified multiple - times, each one causes appropriate output. For example, -o1 -o2 outputs the - substrings matched by the 1st and 2nd capturing parentheses. A separating - string can be specified by --om-separator (default empty). - -19. Improving the first n character searches. - -20. Turn case lists for horizontal and vertical white space into macros so that - they are defined only once. - -21. This set of changes together give more compatible Unicode case-folding - behaviour for characters that have more than one other case when UCP - support is available. - - (a) The Unicode property table now has offsets into a new table of sets of - three or more characters that are case-equivalent. The MultiStage2.py - script that generates these tables (the pcre_ucd.c file) now scans - CaseFolding.txt instead of UnicodeData.txt for character case - information. - - (b) The code for adding characters or ranges of characters to a character - class has been abstracted into a generalized function that also handles - case-independence. In UTF-mode with UCP support, this uses the new data - to handle characters with more than one other case. - - (c) A bug that is fixed as a result of (b) is that codepoints less than 256 - whose other case is greater than 256 are now correctly matched - caselessly. Previously, the high codepoint matched the low one, but not - vice versa. - - (d) The processing of \h, \H, \v, and \ in character classes now makes use - of the new class addition function, using character lists defined as - macros alongside the case definitions of 20 above. - - (e) Caseless back references now work with characters that have more than - one other case. - - (f) General caseless matching of characters with more than one other case - is supported. - -22. Unicode character properties were updated from Unicode 6.2.0 - -23. Improved CMake support under Windows. Patch by Daniel Richard G. - -24. Add support for 32-bit character strings, and UTF-32 - -25. Major JIT compiler update (code refactoring and bugfixing). - Experimental Sparc 32 support is added. - -26. Applied a modified version of Daniel Richard G's patch to create - pcre.h.generic and config.h.generic by "make" instead of in the - PrepareRelease script. - -27. Added a definition for CHAR_NULL (helpful for the z/OS port), and use it in - pcre_compile.c when checking for a zero character. - -28. Introducing a native interface for JIT. Through this interface, the compiled - machine code can be directly executed. The purpose of this interface is to - provide fast pattern matching, so several sanity checks are not performed. - However, feature tests are still performed. The new interface provides - 1.4x speedup compared to the old one. - -29. If pcre_exec() or pcre_dfa_exec() was called with a negative value for - the subject string length, the error given was PCRE_ERROR_BADOFFSET, which - was confusing. There is now a new error PCRE_ERROR_BADLENGTH for this case. - -30. In 8-bit UTF-8 mode, pcretest failed to give an error for data codepoints - greater than 0x7fffffff (which cannot be represented in UTF-8, even under - the "old" RFC 2279). Instead, it ended up passing a negative length to - pcre_exec(). - -31. Add support for GCC's visibility feature to hide internal functions. - -32. Running "pcretest -C pcre8" or "pcretest -C pcre16" gave a spurious error - "unknown -C option" after outputting 0 or 1. - -33. There is now support for generating a code coverage report for the test - suite in environments where gcc is the compiler and lcov is installed. This - is mainly for the benefit of the developers. - -34. If PCRE is built with --enable-valgrind, certain memory regions are marked - unaddressable using valgrind annotations, allowing valgrind to detect - invalid memory accesses. This is mainly for the benefit of the developers. - -25. (*UTF) can now be used to start a pattern in any of the three libraries. - -26. Give configure error if --enable-cpp but no C++ compiler found. - - -Version 8.31 06-July-2012 -------------------------- - -1. Fixing a wrong JIT test case and some compiler warnings. - -2. Removed a bashism from the RunTest script. - -3. Add a cast to pcre_exec.c to fix the warning "unary minus operator applied - to unsigned type, result still unsigned" that was given by an MS compiler - on encountering the code "-sizeof(xxx)". - -4. Partial matching support is added to the JIT compiler. - -5. Fixed several bugs concerned with partial matching of items that consist - of more than one character: - - (a) /^(..)\1/ did not partially match "aba" because checking references was - done on an "all or nothing" basis. This also applied to repeated - references. - - (b) \R did not give a hard partial match if \r was found at the end of the - subject. - - (c) \X did not give a hard partial match after matching one or more - characters at the end of the subject. - - (d) When newline was set to CRLF, a pattern such as /a$/ did not recognize - a partial match for the string "\r". - - (e) When newline was set to CRLF, the metacharacter "." did not recognize - a partial match for a CR character at the end of the subject string. - -6. If JIT is requested using /S++ or -s++ (instead of just /S+ or -s+) when - running pcretest, the text "(JIT)" added to the output whenever JIT is - actually used to run the match. - -7. Individual JIT compile options can be set in pcretest by following -s+[+] - or /S+[+] with a digit between 1 and 7. - -8. OP_NOT now supports any UTF character not just single-byte ones. - -9. (*MARK) control verb is now supported by the JIT compiler. - -10. The command "./RunTest list" lists the available tests without actually - running any of them. (Because I keep forgetting what they all are.) - -11. Add PCRE_INFO_MAXLOOKBEHIND. - -12. Applied a (slightly modified) user-supplied patch that improves performance - when the heap is used for recursion (compiled with --disable-stack-for- - recursion). Instead of malloc and free for each heap frame each time a - logical recursion happens, frames are retained on a chain and re-used where - possible. This sometimes gives as much as 30% improvement. - -13. As documented, (*COMMIT) is now confined to within a recursive subpattern - call. - -14. As documented, (*COMMIT) is now confined to within a positive assertion. - -15. It is now possible to link pcretest with libedit as an alternative to - libreadline. - -16. (*COMMIT) control verb is now supported by the JIT compiler. - -17. The Unicode data tables have been updated to Unicode 6.1.0. - -18. Added --file-list option to pcregrep. - -19. Added binary file support to pcregrep, including the -a, --binary-files, - -I, and --text options. - -20. The madvise function is renamed for posix_madvise for QNX compatibility - reasons. Fixed by Giuseppe D'Angelo. - -21. Fixed a bug for backward assertions with REVERSE 0 in the JIT compiler. - -22. Changed the option for creating symbolic links for 16-bit man pages from - -s to -sf so that re-installing does not cause issues. - -23. Support PCRE_NO_START_OPTIMIZE in JIT as (*MARK) support requires it. - -24. Fixed a very old bug in pcretest that caused errors with restarted DFA - matches in certain environments (the workspace was not being correctly - retained). Also added to pcre_dfa_exec() a simple plausibility check on - some of the workspace data at the beginning of a restart. - -25. \s*\R was auto-possessifying the \s* when it should not, whereas \S*\R - was not doing so when it should - probably a typo introduced by SVN 528 - (change 8.10/14). - -26. When PCRE_UCP was not set, \w+\x{c4} was incorrectly auto-possessifying the - \w+ when the character tables indicated that \x{c4} was a word character. - There were several related cases, all because the tests for doing a table - lookup were testing for characters less than 127 instead of 255. - -27. If a pattern contains capturing parentheses that are not used in a match, - their slots in the ovector are set to -1. For those that are higher than - any matched groups, this happens at the end of processing. In the case when - there were back references that the ovector was too small to contain - (causing temporary malloc'd memory to be used during matching), and the - highest capturing number was not used, memory off the end of the ovector - was incorrectly being set to -1. (It was using the size of the temporary - memory instead of the true size.) - -28. To catch bugs like 27 using valgrind, when pcretest is asked to specify an - ovector size, it uses memory at the end of the block that it has got. - -29. Check for an overlong MARK name and give an error at compile time. The - limit is 255 for the 8-bit library and 65535 for the 16-bit library. - -30. JIT compiler update. - -31. JIT is now supported on jailbroken iOS devices. Thanks for Ruiger - Rill for the patch. - -32. Put spaces around SLJIT_PRINT_D in the JIT compiler. Required by CXX11. - -33. Variable renamings in the PCRE-JIT compiler. No functionality change. - -34. Fixed typos in pcregrep: in two places there was SUPPORT_LIBZ2 instead of - SUPPORT_LIBBZ2. This caused a build problem when bzip2 but not gzip (zlib) - was enabled. - -35. Improve JIT code generation for greedy plus quantifier. - -36. When /((?:a?)*)*c/ or /((?>a?)*)*c/ was matched against "aac", it set group - 1 to "aa" instead of to an empty string. The bug affected repeated groups - that could potentially match an empty string. - -37. Optimizing single character iterators in JIT. - -38. Wide characters specified with \uxxxx in JavaScript mode are now subject to - the same checks as \x{...} characters in non-JavaScript mode. Specifically, - codepoints that are too big for the mode are faulted, and in a UTF mode, - disallowed codepoints are also faulted. - -39. If PCRE was compiled with UTF support, in three places in the DFA - matcher there was code that should only have been obeyed in UTF mode, but - was being obeyed unconditionally. In 8-bit mode this could cause incorrect - processing when bytes with values greater than 127 were present. In 16-bit - mode the bug would be provoked by values in the range 0xfc00 to 0xdc00. In - both cases the values are those that cannot be the first data item in a UTF - character. The three items that might have provoked this were recursions, - possessively repeated groups, and atomic groups. - -40. Ensure that libpcre is explicitly listed in the link commands for pcretest - and pcregrep, because some OS require shared objects to be explicitly - passed to ld, causing the link step to fail if they are not. - -41. There were two incorrect #ifdefs in pcre_study.c, meaning that, in 16-bit - mode, patterns that started with \h* or \R* might be incorrectly matched. - - -Version 8.30 04-February-2012 ------------------------------ - -1. Renamed "isnumber" as "is_a_number" because in some Mac environments this - name is defined in ctype.h. - -2. Fixed a bug in fixed-length calculation for lookbehinds that would show up - only in quite long subpatterns. - -3. Removed the function pcre_info(), which has been obsolete and deprecated - since it was replaced by pcre_fullinfo() in February 2000. - -4. For a non-anchored pattern, if (*SKIP) was given with a name that did not - match a (*MARK), and the match failed at the start of the subject, a - reference to memory before the start of the subject could occur. This bug - was introduced by fix 17 of release 8.21. - -5. A reference to an unset group with zero minimum repetition was giving - totally wrong answers (in non-JavaScript-compatibility mode). For example, - /(another)?(\1?)test/ matched against "hello world test". This bug was - introduced in release 8.13. - -6. Add support for 16-bit character strings (a large amount of work involving - many changes and refactorings). - -7. RunGrepTest failed on msys because \r\n was replaced by whitespace when the - command "pattern=`printf 'xxx\r\njkl'`" was run. The pattern is now taken - from a file. - -8. Ovector size of 2 is also supported by JIT based pcre_exec (the ovector size - rounding is not applied in this particular case). - -9. The invalid Unicode surrogate codepoints U+D800 to U+DFFF are now rejected - if they appear, or are escaped, in patterns. - -10. Get rid of a number of -Wunused-but-set-variable warnings. - -11. The pattern /(?=(*:x))(q|)/ matches an empty string, and returns the mark - "x". The similar pattern /(?=(*:x))((*:y)q|)/ did not return a mark at all. - Oddly, Perl behaves the same way. PCRE has been fixed so that this pattern - also returns the mark "x". This bug applied to capturing parentheses, - non-capturing parentheses, and atomic parentheses. It also applied to some - assertions. - -12. Stephen Kelly's patch to CMakeLists.txt allows it to parse the version - information out of configure.ac instead of relying on pcre.h.generic, which - is not stored in the repository. - -13. Applied Dmitry V. Levin's patch for a more portable method for linking with - -lreadline. - -14. ZH added PCRE_CONFIG_JITTARGET; added its output to pcretest -C. - -15. Applied Graycode's patch to put the top-level frame on the stack rather - than the heap when not using the stack for recursion. This gives a - performance improvement in many cases when recursion is not deep. - -16. Experimental code added to "pcretest -C" to output the stack frame size. - - -Version 8.21 12-Dec-2011 ------------------------- - -1. Updating the JIT compiler. - -2. JIT compiler now supports OP_NCREF, OP_RREF and OP_NRREF. New test cases - are added as well. - -3. Fix cache-flush issue on PowerPC (It is still an experimental JIT port). - PCRE_EXTRA_TABLES is not suported by JIT, and should be checked before - calling _pcre_jit_exec. Some extra comments are added. - -4. (*MARK) settings inside atomic groups that do not contain any capturing - parentheses, for example, (?>a(*:m)), were not being passed out. This bug - was introduced by change 18 for 8.20. - -5. Supporting of \x, \U and \u in JavaScript compatibility mode based on the - ECMA-262 standard. - -6. Lookbehinds such as (?<=a{2}b) that contained a fixed repetition were - erroneously being rejected as "not fixed length" if PCRE_CASELESS was set. - This bug was probably introduced by change 9 of 8.13. - -7. While fixing 6 above, I noticed that a number of other items were being - incorrectly rejected as "not fixed length". This arose partly because newer - opcodes had not been added to the fixed-length checking code. I have (a) - corrected the bug and added tests for these items, and (b) arranged for an - error to occur if an unknown opcode is encountered while checking for fixed - length instead of just assuming "not fixed length". The items that were - rejected were: (*ACCEPT), (*COMMIT), (*FAIL), (*MARK), (*PRUNE), (*SKIP), - (*THEN), \h, \H, \v, \V, and single character negative classes with fixed - repetitions, e.g. [^a]{3}, with and without PCRE_CASELESS. - -8. A possessively repeated conditional subpattern such as (?(?=c)c|d)++ was - being incorrectly compiled and would have given unpredicatble results. - -9. A possessively repeated subpattern with minimum repeat count greater than - one behaved incorrectly. For example, (A){2,}+ behaved as if it was - (A)(A)++ which meant that, after a subsequent mismatch, backtracking into - the first (A) could occur when it should not. - -10. Add a cast and remove a redundant test from the code. - -11. JIT should use pcre_malloc/pcre_free for allocation. - -12. Updated pcre-config so that it no longer shows -L/usr/lib, which seems - best practice nowadays, and helps with cross-compiling. (If the exec_prefix - is anything other than /usr, -L is still shown). - -13. In non-UTF-8 mode, \C is now supported in lookbehinds and DFA matching. - -14. Perl does not support \N without a following name in a [] class; PCRE now - also gives an error. - -15. If a forward reference was repeated with an upper limit of around 2000, - it caused the error "internal error: overran compiling workspace". The - maximum number of forward references (including repeats) was limited by the - internal workspace, and dependent on the LINK_SIZE. The code has been - rewritten so that the workspace expands (via pcre_malloc) if necessary, and - the default depends on LINK_SIZE. There is a new upper limit (for safety) - of around 200,000 forward references. While doing this, I also speeded up - the filling in of repeated forward references. - -16. A repeated forward reference in a pattern such as (a)(?2){2}(.) was - incorrectly expecting the subject to contain another "a" after the start. - -17. When (*SKIP:name) is activated without a corresponding (*MARK:name) earlier - in the match, the SKIP should be ignored. This was not happening; instead - the SKIP was being treated as NOMATCH. For patterns such as - /A(*MARK:A)A+(*SKIP:B)Z|AAC/ this meant that the AAC branch was never - tested. - -18. The behaviour of (*MARK), (*PRUNE), and (*THEN) has been reworked and is - now much more compatible with Perl, in particular in cases where the result - is a non-match for a non-anchored pattern. For example, if - /b(*:m)f|a(*:n)w/ is matched against "abc", the non-match returns the name - "m", where previously it did not return a name. A side effect of this - change is that for partial matches, the last encountered mark name is - returned, as for non matches. A number of tests that were previously not - Perl-compatible have been moved into the Perl-compatible test files. The - refactoring has had the pleasing side effect of removing one argument from - the match() function, thus reducing its stack requirements. - -19. If the /S+ option was used in pcretest to study a pattern using JIT, - subsequent uses of /S (without +) incorrectly behaved like /S+. - -21. Retrieve executable code size support for the JIT compiler and fixing - some warnings. - -22. A caseless match of a UTF-8 character whose other case uses fewer bytes did - not work when the shorter character appeared right at the end of the - subject string. - -23. Added some (int) casts to non-JIT modules to reduce warnings on 64-bit - systems. - -24. Added PCRE_INFO_JITSIZE to pass on the value from (21) above, and also - output it when the /M option is used in pcretest. - -25. The CheckMan script was not being included in the distribution. Also, added - an explicit "perl" to run Perl scripts from the PrepareRelease script - because this is reportedly needed in Windows. - -26. If study data was being save in a file and studying had not found a set of - "starts with" bytes for the pattern, the data written to the file (though - never used) was taken from uninitialized memory and so caused valgrind to - complain. - -27. Updated RunTest.bat as provided by Sheri Pierce. - -28. Fixed a possible uninitialized memory bug in pcre_jit_compile.c. - -29. Computation of memory usage for the table of capturing group names was - giving an unnecessarily large value. - - -Version 8.20 21-Oct-2011 ------------------------- - -1. Change 37 of 8.13 broke patterns like [:a]...[b:] because it thought it had - a POSIX class. After further experiments with Perl, which convinced me that - Perl has bugs and confusions, a closing square bracket is no longer allowed - in a POSIX name. This bug also affected patterns with classes that started - with full stops. - -2. If a pattern such as /(a)b|ac/ is matched against "ac", there is no - captured substring, but while checking the failing first alternative, - substring 1 is temporarily captured. If the output vector supplied to - pcre_exec() was not big enough for this capture, the yield of the function - was still zero ("insufficient space for captured substrings"). This cannot - be totally fixed without adding another stack variable, which seems a lot - of expense for a edge case. However, I have improved the situation in cases - such as /(a)(b)x|abc/ matched against "abc", where the return code - indicates that fewer than the maximum number of slots in the ovector have - been set. - -3. Related to (2) above: when there are more back references in a pattern than - slots in the output vector, pcre_exec() uses temporary memory during - matching, and copies in the captures as far as possible afterwards. It was - using the entire output vector, but this conflicts with the specification - that only 2/3 is used for passing back captured substrings. Now it uses - only the first 2/3, for compatibility. This is, of course, another edge - case. - -4. Zoltan Herczeg's just-in-time compiler support has been integrated into the - main code base, and can be used by building with --enable-jit. When this is - done, pcregrep automatically uses it unless --disable-pcregrep-jit or the - runtime --no-jit option is given. - -5. When the number of matches in a pcre_dfa_exec() run exactly filled the - ovector, the return from the function was zero, implying that there were - other matches that did not fit. The correct "exactly full" value is now - returned. - -6. If a subpattern that was called recursively or as a subroutine contained - (*PRUNE) or any other control that caused it to give a non-standard return, - invalid errors such as "Error -26 (nested recursion at the same subject - position)" or even infinite loops could occur. - -7. If a pattern such as /a(*SKIP)c|b(*ACCEPT)|/ was studied, it stopped - computing the minimum length on reaching *ACCEPT, and so ended up with the - wrong value of 1 rather than 0. Further investigation indicates that - computing a minimum subject length in the presence of *ACCEPT is difficult - (think back references, subroutine calls), and so I have changed the code - so that no minimum is registered for a pattern that contains *ACCEPT. - -8. If (*THEN) was present in the first (true) branch of a conditional group, - it was not handled as intended. [But see 16 below.] - -9. Replaced RunTest.bat and CMakeLists.txt with improved versions provided by - Sheri Pierce. - -10. A pathological pattern such as /(*ACCEPT)a/ was miscompiled, thinking that - the first byte in a match must be "a". - -11. Change 17 for 8.13 increased the recursion depth for patterns like - /a(?:.)*?a/ drastically. I've improved things by remembering whether a - pattern contains any instances of (*THEN). If it does not, the old - optimizations are restored. It would be nice to do this on a per-group - basis, but at the moment that is not feasible. - -12. In some environments, the output of pcretest -C is CRLF terminated. This - broke RunTest's code that checks for the link size. A single white space - character after the value is now allowed for. - -13. RunTest now checks for the "fr" locale as well as for "fr_FR" and "french". - For "fr", it uses the Windows-specific input and output files. - -14. If (*THEN) appeared in a group that was called recursively or as a - subroutine, it did not work as intended. [But see next item.] - -15. Consider the pattern /A (B(*THEN)C) | D/ where A, B, C, and D are complex - pattern fragments (but not containing any | characters). If A and B are - matched, but there is a failure in C so that it backtracks to (*THEN), PCRE - was behaving differently to Perl. PCRE backtracked into A, but Perl goes to - D. In other words, Perl considers parentheses that do not contain any | - characters to be part of a surrounding alternative, whereas PCRE was - treading (B(*THEN)C) the same as (B(*THEN)C|(*FAIL)) -- which Perl handles - differently. PCRE now behaves in the same way as Perl, except in the case - of subroutine/recursion calls such as (?1) which have in any case always - been different (but PCRE had them first :-). - -16. Related to 15 above: Perl does not treat the | in a conditional group as - creating alternatives. Such a group is treated in the same way as an - ordinary group without any | characters when processing (*THEN). PCRE has - been changed to match Perl's behaviour. - -17. If a user had set PCREGREP_COLO(U)R to something other than 1:31, the - RunGrepTest script failed. - -18. Change 22 for version 13 caused atomic groups to use more stack. This is - inevitable for groups that contain captures, but it can lead to a lot of - stack use in large patterns. The old behaviour has been restored for atomic - groups that do not contain any capturing parentheses. - -19. If the PCRE_NO_START_OPTIMIZE option was set for pcre_compile(), it did not - suppress the check for a minimum subject length at run time. (If it was - given to pcre_exec() or pcre_dfa_exec() it did work.) - -20. Fixed an ASCII-dependent infelicity in pcretest that would have made it - fail to work when decoding hex characters in data strings in EBCDIC - environments. - -21. It appears that in at least one Mac OS environment, the isxdigit() function - is implemented as a macro that evaluates to its argument more than once, - contravening the C 90 Standard (I haven't checked a later standard). There - was an instance in pcretest which caused it to go wrong when processing - \x{...} escapes in subject strings. The has been rewritten to avoid using - things like p++ in the argument of isxdigit(). - - -Version 8.13 16-Aug-2011 ------------------------- - -1. The Unicode data tables have been updated to Unicode 6.0.0. - -2. Two minor typos in pcre_internal.h have been fixed. - -3. Added #include <string.h> to pcre_scanner_unittest.cc, pcrecpp.cc, and - pcrecpp_unittest.cc. They are needed for strcmp(), memset(), and strchr() - in some environments (e.g. Solaris 10/SPARC using Sun Studio 12U2). - -4. There were a number of related bugs in the code for matching backrefences - caselessly in UTF-8 mode when codes for the characters concerned were - different numbers of bytes. For example, U+023A and U+2C65 are an upper - and lower case pair, using 2 and 3 bytes, respectively. The main bugs were: - (a) A reference to 3 copies of a 2-byte code matched only 2 of a 3-byte - code. (b) A reference to 2 copies of a 3-byte code would not match 2 of a - 2-byte code at the end of the subject (it thought there wasn't enough data - left). - -5. Comprehensive information about what went wrong is now returned by - pcre_exec() and pcre_dfa_exec() when the UTF-8 string check fails, as long - as the output vector has at least 2 elements. The offset of the start of - the failing character and a reason code are placed in the vector. - -6. When the UTF-8 string check fails for pcre_compile(), the offset that is - now returned is for the first byte of the failing character, instead of the - last byte inspected. This is an incompatible change, but I hope it is small - enough not to be a problem. It makes the returned offset consistent with - pcre_exec() and pcre_dfa_exec(). - -7. pcretest now gives a text phrase as well as the error number when - pcre_exec() or pcre_dfa_exec() fails; if the error is a UTF-8 check - failure, the offset and reason code are output. - -8. When \R was used with a maximizing quantifier it failed to skip backwards - over a \r\n pair if the subsequent match failed. Instead, it just skipped - back over a single character (\n). This seems wrong (because it treated the - two characters as a single entity when going forwards), conflicts with the - documentation that \R is equivalent to (?>\r\n|\n|...etc), and makes the - behaviour of \R* different to (\R)*, which also seems wrong. The behaviour - has been changed. - -9. Some internal refactoring has changed the processing so that the handling - of the PCRE_CASELESS and PCRE_MULTILINE options is done entirely at compile - time (the PCRE_DOTALL option was changed this way some time ago: version - 7.7 change 16). This has made it possible to abolish the OP_OPT op code, - which was always a bit of a fudge. It also means that there is one less - argument for the match() function, which reduces its stack requirements - slightly. This change also fixes an incompatibility with Perl: the pattern - (?i:([^b]))(?1) should not match "ab", but previously PCRE gave a match. - -10. More internal refactoring has drastically reduced the number of recursive - calls to match() for possessively repeated groups such as (abc)++ when - using pcre_exec(). - -11. While implementing 10, a number of bugs in the handling of groups were - discovered and fixed: - - (?<=(a)+) was not diagnosed as invalid (non-fixed-length lookbehind). - (a|)*(?1) gave a compile-time internal error. - ((a|)+)+ did not notice that the outer group could match an empty string. - (^a|^)+ was not marked as anchored. - (.*a|.*)+ was not marked as matching at start or after a newline. - -12. Yet more internal refactoring has removed another argument from the match() - function. Special calls to this function are now indicated by setting a - value in a variable in the "match data" data block. - -13. Be more explicit in pcre_study() instead of relying on "default" for - opcodes that mean there is no starting character; this means that when new - ones are added and accidentally left out of pcre_study(), testing should - pick them up. - -14. The -s option of pcretest has been documented for ages as being an old - synonym of -m (show memory usage). I have changed it to mean "force study - for every regex", that is, assume /S for every regex. This is similar to -i - and -d etc. It's slightly incompatible, but I'm hoping nobody is still - using it. It makes it easier to run collections of tests with and without - study enabled, and thereby test pcre_study() more easily. All the standard - tests are now run with and without -s (but some patterns can be marked as - "never study" - see 20 below). - -15. When (*ACCEPT) was used in a subpattern that was called recursively, the - restoration of the capturing data to the outer values was not happening - correctly. - -16. If a recursively called subpattern ended with (*ACCEPT) and matched an - empty string, and PCRE_NOTEMPTY was set, pcre_exec() thought the whole - pattern had matched an empty string, and so incorrectly returned a no - match. - -17. There was optimizing code for the last branch of non-capturing parentheses, - and also for the obeyed branch of a conditional subexpression, which used - tail recursion to cut down on stack usage. Unfortunately, now that there is - the possibility of (*THEN) occurring in these branches, tail recursion is - no longer possible because the return has to be checked for (*THEN). These - two optimizations have therefore been removed. [But see 8.20/11 above.] - -18. If a pattern containing \R was studied, it was assumed that \R always - matched two bytes, thus causing the minimum subject length to be - incorrectly computed because \R can also match just one byte. - -19. If a pattern containing (*ACCEPT) was studied, the minimum subject length - was incorrectly computed. - -20. If /S is present twice on a test pattern in pcretest input, it now - *disables* studying, thereby overriding the use of -s on the command line - (see 14 above). This is necessary for one or two tests to keep the output - identical in both cases. - -21. When (*ACCEPT) was used in an assertion that matched an empty string and - PCRE_NOTEMPTY was set, PCRE applied the non-empty test to the assertion. - -22. When an atomic group that contained a capturing parenthesis was - successfully matched, but the branch in which it appeared failed, the - capturing was not being forgotten if a higher numbered group was later - captured. For example, /(?>(a))b|(a)c/ when matching "ac" set capturing - group 1 to "a", when in fact it should be unset. This applied to multi- - branched capturing and non-capturing groups, repeated or not, and also to - positive assertions (capturing in negative assertions does not happen - in PCRE) and also to nested atomic groups. - -23. Add the ++ qualifier feature to pcretest, to show the remainder of the - subject after a captured substring, to make it easier to tell which of a - number of identical substrings has been captured. - -24. The way atomic groups are processed by pcre_exec() has been changed so that - if they are repeated, backtracking one repetition now resets captured - values correctly. For example, if ((?>(a+)b)+aabab) is matched against - "aaaabaaabaabab" the value of captured group 2 is now correctly recorded as - "aaa". Previously, it would have been "a". As part of this code - refactoring, the way recursive calls are handled has also been changed. - -25. If an assertion condition captured any substrings, they were not passed - back unless some other capturing happened later. For example, if - (?(?=(a))a) was matched against "a", no capturing was returned. - -26. When studying a pattern that contained subroutine calls or assertions, - the code for finding the minimum length of a possible match was handling - direct recursions such as (xxx(?1)|yyy) but not mutual recursions (where - group 1 called group 2 while simultaneously a separate group 2 called group - 1). A stack overflow occurred in this case. I have fixed this by limiting - the recursion depth to 10. - -27. Updated RunTest.bat in the distribution to the version supplied by Tom - Fortmann. This supports explicit test numbers on the command line, and has - argument validation and error reporting. - -28. An instance of \X with an unlimited repeat could fail if at any point the - first character it looked at was a mark character. - -29. Some minor code refactoring concerning Unicode properties and scripts - should reduce the stack requirement of match() slightly. - -30. Added the '=' option to pcretest to check the setting of unused capturing - slots at the end of the pattern, which are documented as being -1, but are - not included in the return count. - -31. If \k was not followed by a braced, angle-bracketed, or quoted name, PCRE - compiled something random. Now it gives a compile-time error (as does - Perl). - -32. A *MARK encountered during the processing of a positive assertion is now - recorded and passed back (compatible with Perl). - -33. If --only-matching or --colour was set on a pcregrep call whose pattern - had alternative anchored branches, the search for a second match in a line - was done as if at the line start. Thus, for example, /^01|^02/ incorrectly - matched the line "0102" twice. The same bug affected patterns that started - with a backwards assertion. For example /\b01|\b02/ also matched "0102" - twice. - -34. Previously, PCRE did not allow quantification of assertions. However, Perl - does, and because of capturing effects, quantifying parenthesized - assertions may at times be useful. Quantifiers are now allowed for - parenthesized assertions. - -35. A minor code tidy in pcre_compile() when checking options for \R usage. - -36. \g was being checked for fancy things in a character class, when it should - just be a literal "g". - -37. PCRE was rejecting [:a[:digit:]] whereas Perl was not. It seems that the - appearance of a nested POSIX class supersedes an apparent external class. - For example, [:a[:digit:]b:] matches "a", "b", ":", or a digit. Also, - unescaped square brackets may also appear as part of class names. For - example, [:a[:abc]b:] gives unknown class "[:abc]b:]". PCRE now behaves - more like Perl. (But see 8.20/1 above.) - -38. PCRE was giving an error for \N with a braced quantifier such as {1,} (this - was because it thought it was \N{name}, which is not supported). - -39. Add minix to OS list not supporting the -S option in pcretest. - -40. PCRE tries to detect cases of infinite recursion at compile time, but it - cannot analyze patterns in sufficient detail to catch mutual recursions - such as ((?1))((?2)). There is now a runtime test that gives an error if a - subgroup is called recursively as a subpattern for a second time at the - same position in the subject string. In previous releases this might have - been caught by the recursion limit, or it might have run out of stack. - -41. A pattern such as /(?(R)a+|(?R)b)/ is quite safe, as the recursion can - happen only once. PCRE was, however incorrectly giving a compile time error - "recursive call could loop indefinitely" because it cannot analyze the - pattern in sufficient detail. The compile time test no longer happens when - PCRE is compiling a conditional subpattern, but actual runaway loops are - now caught at runtime (see 40 above). - -42. It seems that Perl allows any characters other than a closing parenthesis - to be part of the NAME in (*MARK:NAME) and other backtracking verbs. PCRE - has been changed to be the same. - -43. Updated configure.ac to put in more quoting round AC_LANG_PROGRAM etc. so - as not to get warnings when autogen.sh is called. Also changed - AC_PROG_LIBTOOL (deprecated) to LT_INIT (the current macro). - -44. To help people who use pcregrep to scan files containing exceedingly long - lines, the following changes have been made: - - (a) The default value of the buffer size parameter has been increased from - 8K to 20K. (The actual buffer used is three times this size.) - - (b) The default can be changed by ./configure --with-pcregrep-bufsize when - PCRE is built. - - (c) A --buffer-size=n option has been added to pcregrep, to allow the size - to be set at run time. - - (d) Numerical values in pcregrep options can be followed by K or M, for - example --buffer-size=50K. - - (e) If a line being scanned overflows pcregrep's buffer, an error is now - given and the return code is set to 2. - -45. Add a pointer to the latest mark to the callout data block. - -46. The pattern /.(*F)/, when applied to "abc" with PCRE_PARTIAL_HARD, gave a - partial match of an empty string instead of no match. This was specific to - the use of ".". - -47. The pattern /f.*/8s, when applied to "for" with PCRE_PARTIAL_HARD, gave a - complete match instead of a partial match. This bug was dependent on both - the PCRE_UTF8 and PCRE_DOTALL options being set. - -48. For a pattern such as /\babc|\bdef/ pcre_study() was failing to set up the - starting byte set, because \b was not being ignored. - - -Version 8.12 15-Jan-2011 ------------------------- - -1. Fixed some typos in the markup of the man pages, and wrote a script that - checks for such things as part of the documentation building process. - -2. On a big-endian 64-bit system, pcregrep did not correctly process the - --match-limit and --recursion-limit options (added for 8.11). In - particular, this made one of the standard tests fail. (The integer value - went into the wrong half of a long int.) - -3. If the --colour option was given to pcregrep with -v (invert match), it - did strange things, either producing crazy output, or crashing. It should, - of course, ignore a request for colour when reporting lines that do not - match. - -4. Another pcregrep bug caused similar problems if --colour was specified with - -M (multiline) and the pattern match finished with a line ending. - -5. In pcregrep, when a pattern that ended with a literal newline sequence was - matched in multiline mode, the following line was shown as part of the - match. This seems wrong, so I have changed it. - -6. Another pcregrep bug in multiline mode, when --colour was specified, caused - the check for further matches in the same line (so they could be coloured) - to overrun the end of the current line. If another match was found, it was - incorrectly shown (and then shown again when found in the next line). - -7. If pcregrep was compiled under Windows, there was a reference to the - function pcregrep_exit() before it was defined. I am assuming this was - the cause of the "error C2371: 'pcregrep_exit' : redefinition;" that was - reported by a user. I've moved the definition above the reference. - - -Version 8.11 10-Dec-2010 ------------------------- - -1. (*THEN) was not working properly if there were untried alternatives prior - to it in the current branch. For example, in ((a|b)(*THEN)(*F)|c..) it - backtracked to try for "b" instead of moving to the next alternative branch - at the same level (in this case, to look for "c"). The Perl documentation - is clear that when (*THEN) is backtracked onto, it goes to the "next - alternative in the innermost enclosing group". - -2. (*COMMIT) was not overriding (*THEN), as it does in Perl. In a pattern - such as (A(*COMMIT)B(*THEN)C|D) any failure after matching A should - result in overall failure. Similarly, (*COMMIT) now overrides (*PRUNE) and - (*SKIP), (*SKIP) overrides (*PRUNE) and (*THEN), and (*PRUNE) overrides - (*THEN). - -3. If \s appeared in a character class, it removed the VT character from - the class, even if it had been included by some previous item, for example - in [\x00-\xff\s]. (This was a bug related to the fact that VT is not part - of \s, but is part of the POSIX "space" class.) - -4. A partial match never returns an empty string (because you can always - match an empty string at the end of the subject); however the checking for - an empty string was starting at the "start of match" point. This has been - changed to the "earliest inspected character" point, because the returned - data for a partial match starts at this character. This means that, for - example, /(?<=abc)def/ gives a partial match for the subject "abc" - (previously it gave "no match"). - -5. Changes have been made to the way PCRE_PARTIAL_HARD affects the matching - of $, \z, \Z, \b, and \B. If the match point is at the end of the string, - previously a full match would be given. However, setting PCRE_PARTIAL_HARD - has an implication that the given string is incomplete (because a partial - match is preferred over a full match). For this reason, these items now - give a partial match in this situation. [Aside: previously, the one case - /t\b/ matched against "cat" with PCRE_PARTIAL_HARD set did return a partial - match rather than a full match, which was wrong by the old rules, but is - now correct.] - -6. There was a bug in the handling of #-introduced comments, recognized when - PCRE_EXTENDED is set, when PCRE_NEWLINE_ANY and PCRE_UTF8 were also set. - If a UTF-8 multi-byte character included the byte 0x85 (e.g. +U0445, whose - UTF-8 encoding is 0xd1,0x85), this was misinterpreted as a newline when - scanning for the end of the comment. (*Character* 0x85 is an "any" newline, - but *byte* 0x85 is not, in UTF-8 mode). This bug was present in several - places in pcre_compile(). - -7. Related to (6) above, when pcre_compile() was skipping #-introduced - comments when looking ahead for named forward references to subpatterns, - the only newline sequence it recognized was NL. It now handles newlines - according to the set newline convention. - -8. SunOS4 doesn't have strerror() or strtoul(); pcregrep dealt with the - former, but used strtoul(), whereas pcretest avoided strtoul() but did not - cater for a lack of strerror(). These oversights have been fixed. - -9. Added --match-limit and --recursion-limit to pcregrep. - -10. Added two casts needed to build with Visual Studio when NO_RECURSE is set. - -11. When the -o option was used, pcregrep was setting a return code of 1, even - when matches were found, and --line-buffered was not being honoured. - -12. Added an optional parentheses number to the -o and --only-matching options - of pcregrep. - -13. Imitating Perl's /g action for multiple matches is tricky when the pattern - can match an empty string. The code to do it in pcretest and pcredemo - needed fixing: - - (a) When the newline convention was "crlf", pcretest got it wrong, skipping - only one byte after an empty string match just before CRLF (this case - just got forgotten; "any" and "anycrlf" were OK). - - (b) The pcretest code also had a bug, causing it to loop forever in UTF-8 - mode when an empty string match preceded an ASCII character followed by - a non-ASCII character. (The code for advancing by one character rather - than one byte was nonsense.) - - (c) The pcredemo.c sample program did not have any code at all to handle - the cases when CRLF is a valid newline sequence. - -14. Neither pcre_exec() nor pcre_dfa_exec() was checking that the value given - as a starting offset was within the subject string. There is now a new - error, PCRE_ERROR_BADOFFSET, which is returned if the starting offset is - negative or greater than the length of the string. In order to test this, - pcretest is extended to allow the setting of negative starting offsets. - -15. In both pcre_exec() and pcre_dfa_exec() the code for checking that the - starting offset points to the beginning of a UTF-8 character was - unnecessarily clumsy. I tidied it up. - -16. Added PCRE_ERROR_SHORTUTF8 to make it possible to distinguish between a - bad UTF-8 sequence and one that is incomplete when using PCRE_PARTIAL_HARD. - -17. Nobody had reported that the --include_dir option, which was added in - release 7.7 should have been called --include-dir (hyphen, not underscore) - for compatibility with GNU grep. I have changed it to --include-dir, but - left --include_dir as an undocumented synonym, and the same for - --exclude-dir, though that is not available in GNU grep, at least as of - release 2.5.4. - -18. At a user's suggestion, the macros GETCHAR and friends (which pick up UTF-8 - characters from a string of bytes) have been redefined so as not to use - loops, in order to improve performance in some environments. At the same - time, I abstracted some of the common code into auxiliary macros to save - repetition (this should not affect the compiled code). - -19. If \c was followed by a multibyte UTF-8 character, bad things happened. A - compile-time error is now given if \c is not followed by an ASCII - character, that is, a byte less than 128. (In EBCDIC mode, the code is - different, and any byte value is allowed.) - -20. Recognize (*NO_START_OPT) at the start of a pattern to set the PCRE_NO_ - START_OPTIMIZE option, which is now allowed at compile time - but just - passed through to pcre_exec() or pcre_dfa_exec(). This makes it available - to pcregrep and other applications that have no direct access to PCRE - options. The new /Y option in pcretest sets this option when calling - pcre_compile(). - -21. Change 18 of release 8.01 broke the use of named subpatterns for recursive - back references. Groups containing recursive back references were forced to - be atomic by that change, but in the case of named groups, the amount of - memory required was incorrectly computed, leading to "Failed: internal - error: code overflow". This has been fixed. - -22. Some patches to pcre_stringpiece.h, pcre_stringpiece_unittest.cc, and - pcretest.c, to avoid build problems in some Borland environments. - - -Version 8.10 25-Jun-2010 ------------------------- - -1. Added support for (*MARK:ARG) and for ARG additions to PRUNE, SKIP, and - THEN. - -2. (*ACCEPT) was not working when inside an atomic group. - -3. Inside a character class, \B is treated as a literal by default, but - faulted if PCRE_EXTRA is set. This mimics Perl's behaviour (the -w option - causes the error). The code is unchanged, but I tidied the documentation. - -4. Inside a character class, PCRE always treated \R and \X as literals, - whereas Perl faults them if its -w option is set. I have changed PCRE so - that it faults them when PCRE_EXTRA is set. - -5. Added support for \N, which always matches any character other than - newline. (It is the same as "." when PCRE_DOTALL is not set.) - -6. When compiling pcregrep with newer versions of gcc which may have - FORTIFY_SOURCE set, several warnings "ignoring return value of 'fwrite', - declared with attribute warn_unused_result" were given. Just casting the - result to (void) does not stop the warnings; a more elaborate fudge is - needed. I've used a macro to implement this. - -7. Minor change to pcretest.c to avoid a compiler warning. - -8. Added four artifical Unicode properties to help with an option to make - \s etc use properties (see next item). The new properties are: Xan - (alphanumeric), Xsp (Perl space), Xps (POSIX space), and Xwd (word). - -9. Added PCRE_UCP to make \b, \d, \s, \w, and certain POSIX character classes - use Unicode properties. (*UCP) at the start of a pattern can be used to set - this option. Modified pcretest to add /W to test this facility. Added - REG_UCP to make it available via the POSIX interface. - -10. Added --line-buffered to pcregrep. - -11. In UTF-8 mode, if a pattern that was compiled with PCRE_CASELESS was - studied, and the match started with a letter with a code point greater than - 127 whose first byte was different to the first byte of the other case of - the letter, the other case of this starting letter was not recognized - (#976). - -12. If a pattern that was studied started with a repeated Unicode property - test, for example, \p{Nd}+, there was the theoretical possibility of - setting up an incorrect bitmap of starting bytes, but fortunately it could - not have actually happened in practice until change 8 above was made (it - added property types that matched character-matching opcodes). - -13. pcre_study() now recognizes \h, \v, and \R when constructing a bit map of - possible starting bytes for non-anchored patterns. - -14. Extended the "auto-possessify" feature of pcre_compile(). It now recognizes - \R, and also a number of cases that involve Unicode properties, both - explicit and implicit when PCRE_UCP is set. - -15. If a repeated Unicode property match (e.g. \p{Lu}*) was used with non-UTF-8 - input, it could crash or give wrong results if characters with values - greater than 0xc0 were present in the subject string. (Detail: it assumed - UTF-8 input when processing these items.) - -16. Added a lot of (int) casts to avoid compiler warnings in systems where - size_t is 64-bit (#991). - -17. Added a check for running out of memory when PCRE is compiled with - --disable-stack-for-recursion (#990). - -18. If the last data line in a file for pcretest does not have a newline on - the end, a newline was missing in the output. - -19. The default pcre_chartables.c file recognizes only ASCII characters (values - less than 128) in its various bitmaps. However, there is a facility for - generating tables according to the current locale when PCRE is compiled. It - turns out that in some environments, 0x85 and 0xa0, which are Unicode space - characters, are recognized by isspace() and therefore were getting set in - these tables, and indeed these tables seem to approximate to ISO 8859. This - caused a problem in UTF-8 mode when pcre_study() was used to create a list - of bytes that can start a match. For \s, it was including 0x85 and 0xa0, - which of course cannot start UTF-8 characters. I have changed the code so - that only real ASCII characters (less than 128) and the correct starting - bytes for UTF-8 encodings are set for characters greater than 127 when in - UTF-8 mode. (When PCRE_UCP is set - see 9 above - the code is different - altogether.) - -20. Added the /T option to pcretest so as to be able to run tests with non- - standard character tables, thus making it possible to include the tests - used for 19 above in the standard set of tests. - -21. A pattern such as (?&t)(?#()(?(DEFINE)(?<t>a)) which has a forward - reference to a subpattern the other side of a comment that contains an - opening parenthesis caused either an internal compiling error, or a - reference to the wrong subpattern. - - -Version 8.02 19-Mar-2010 ------------------------- - -1. The Unicode data tables have been updated to Unicode 5.2.0. - -2. Added the option --libs-cpp to pcre-config, but only when C++ support is - configured. - -3. Updated the licensing terms in the pcregexp.pas file, as agreed with the - original author of that file, following a query about its status. - -4. On systems that do not have stdint.h (e.g. Solaris), check for and include - inttypes.h instead. This fixes a bug that was introduced by change 8.01/8. - -5. A pattern such as (?&t)*+(?(DEFINE)(?<t>.)) which has a possessive - quantifier applied to a forward-referencing subroutine call, could compile - incorrect code or give the error "internal error: previously-checked - referenced subpattern not found". - -6. Both MS Visual Studio and Symbian OS have problems with initializing - variables to point to external functions. For these systems, therefore, - pcre_malloc etc. are now initialized to local functions that call the - relevant global functions. - -7. There were two entries missing in the vectors called coptable and poptable - in pcre_dfa_exec.c. This could lead to memory accesses outsize the vectors. - I've fixed the data, and added a kludgy way of testing at compile time that - the lengths are correct (equal to the number of opcodes). - -8. Following on from 7, I added a similar kludge to check the length of the - eint vector in pcreposix.c. - -9. Error texts for pcre_compile() are held as one long string to avoid too - much relocation at load time. To find a text, the string is searched, - counting zeros. There was no check for running off the end of the string, - which could happen if a new error number was added without updating the - string. - -10. \K gave a compile-time error if it appeared in a lookbehind assersion. - -11. \K was not working if it appeared in an atomic group or in a group that - was called as a "subroutine", or in an assertion. Perl 5.11 documents that - \K is "not well defined" if used in an assertion. PCRE now accepts it if - the assertion is positive, but not if it is negative. - -12. Change 11 fortuitously reduced the size of the stack frame used in the - "match()" function of pcre_exec.c by one pointer. Forthcoming - implementation of support for (*MARK) will need an extra pointer on the - stack; I have reserved it now, so that the stack frame size does not - decrease. - -13. A pattern such as (?P<L1>(?P<L2>0)|(?P>L2)(?P>L1)) in which the only other - item in branch that calls a recursion is a subroutine call - as in the - second branch in the above example - was incorrectly given the compile- - time error "recursive call could loop indefinitely" because pcre_compile() - was not correctly checking the subroutine for matching a non-empty string. - -14. The checks for overrunning compiling workspace could trigger after an - overrun had occurred. This is a "should never occur" error, but it can be - triggered by pathological patterns such as hundreds of nested parentheses. - The checks now trigger 100 bytes before the end of the workspace. - -15. Fix typo in configure.ac: "srtoq" should be "strtoq". - - -Version 8.01 19-Jan-2010 ------------------------- - -1. If a pattern contained a conditional subpattern with only one branch (in - particular, this includes all (*DEFINE) patterns), a call to pcre_study() - computed the wrong minimum data length (which is of course zero for such - subpatterns). This could cause incorrect "no match" results. - -2. For patterns such as (?i)a(?-i)b|c where an option setting at the start of - the pattern is reset in the first branch, pcre_compile() failed with - "internal error: code overflow at offset...". This happened only when - the reset was to the original external option setting. (An optimization - abstracts leading options settings into an external setting, which was the - cause of this.) - -3. A pattern such as ^(?!a(*SKIP)b) where a negative assertion contained one - of the verbs SKIP, PRUNE, or COMMIT, did not work correctly. When the - assertion pattern did not match (meaning that the assertion was true), it - was incorrectly treated as false if the SKIP had been reached during the - matching. This also applied to assertions used as conditions. - -4. If an item that is not supported by pcre_dfa_exec() was encountered in an - assertion subpattern, including such a pattern used as a condition, - unpredictable results occurred, instead of the error return - PCRE_ERROR_DFA_UITEM. - -5. The C++ GlobalReplace function was not working like Perl for the special - situation when an empty string is matched. It now does the fancy magic - stuff that is necessary. - -6. In pcre_internal.h, obsolete includes to setjmp.h and stdarg.h have been - removed. (These were left over from very, very early versions of PCRE.) - -7. Some cosmetic changes to the code to make life easier when compiling it - as part of something else: - - (a) Change DEBUG to PCRE_DEBUG. - - (b) In pcre_compile(), rename the member of the "branch_chain" structure - called "current" as "current_branch", to prevent a collision with the - Linux macro when compiled as a kernel module. - - (c) In pcre_study(), rename the function set_bit() as set_table_bit(), to - prevent a collision with the Linux macro when compiled as a kernel - module. - -8. In pcre_compile() there are some checks for integer overflows that used to - cast potentially large values to (double). This has been changed to that - when building, a check for int64_t is made, and if it is found, it is used - instead, thus avoiding the use of floating point arithmetic. (There is no - other use of FP in PCRE.) If int64_t is not found, the fallback is to - double. - -9. Added two casts to avoid signed/unsigned warnings from VS Studio Express - 2005 (difference between two addresses compared to an unsigned value). - -10. Change the standard AC_CHECK_LIB test for libbz2 in configure.ac to a - custom one, because of the following reported problem in Windows: - - - libbz2 uses the Pascal calling convention (WINAPI) for the functions - under Win32. - - The standard autoconf AC_CHECK_LIB fails to include "bzlib.h", - therefore missing the function definition. - - The compiler thus generates a "C" signature for the test function. - - The linker fails to find the "C" function. - - PCRE fails to configure if asked to do so against libbz2. - -11. When running libtoolize from libtool-2.2.6b as part of autogen.sh, these - messages were output: - - Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.ac and - rerunning libtoolize, to keep the correct libtool macros in-tree. - Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am. - - I have done both of these things. - -12. Although pcre_dfa_exec() does not use nearly as much stack as pcre_exec() - most of the time, it *can* run out if it is given a pattern that contains a - runaway infinite recursion. I updated the discussion in the pcrestack man - page. - -13. Now that we have gone to the x.xx style of version numbers, the minor - version may start with zero. Using 08 or 09 is a bad idea because users - might check the value of PCRE_MINOR in their code, and 08 or 09 may be - interpreted as invalid octal numbers. I've updated the previous comment in - configure.ac, and also added a check that gives an error if 08 or 09 are - used. - -14. Change 8.00/11 was not quite complete: code had been accidentally omitted, - causing partial matching to fail when the end of the subject matched \W - in a UTF-8 pattern where \W was quantified with a minimum of 3. - -15. There were some discrepancies between the declarations in pcre_internal.h - of _pcre_is_newline(), _pcre_was_newline(), and _pcre_valid_utf8() and - their definitions. The declarations used "const uschar *" and the - definitions used USPTR. Even though USPTR is normally defined as "const - unsigned char *" (and uschar is typedeffed as "unsigned char"), it was - reported that: "This difference in casting confuses some C++ compilers, for - example, SunCC recognizes above declarations as different functions and - generates broken code for hbpcre." I have changed the declarations to use - USPTR. - -16. GNU libtool is named differently on some systems. The autogen.sh script now - tries several variants such as glibtoolize (MacOSX) and libtoolize1x - (FreeBSD). - -17. Applied Craig's patch that fixes an HP aCC compile error in pcre 8.00 - (strtoXX undefined when compiling pcrecpp.cc). The patch contains this - comment: "Figure out how to create a longlong from a string: strtoll and - equivalent. It's not enough to call AC_CHECK_FUNCS: hpux has a strtoll, for - instance, but it only takes 2 args instead of 3!" - -18. A subtle bug concerned with back references has been fixed by a change of - specification, with a corresponding code fix. A pattern such as - ^(xa|=?\1a)+$ which contains a back reference inside the group to which it - refers, was giving matches when it shouldn't. For example, xa=xaaa would - match that pattern. Interestingly, Perl (at least up to 5.11.3) has the - same bug. Such groups have to be quantified to be useful, or contained - inside another quantified group. (If there's no repetition, the reference - can never match.) The problem arises because, having left the group and - moved on to the rest of the pattern, a later failure that backtracks into - the group uses the captured value from the final iteration of the group - rather than the correct earlier one. I have fixed this in PCRE by forcing - any group that contains a reference to itself to be an atomic group; that - is, there cannot be any backtracking into it once it has completed. This is - similar to recursive and subroutine calls. - - -Version 8.00 19-Oct-09 ----------------------- - -1. The table for translating pcre_compile() error codes into POSIX error codes - was out-of-date, and there was no check on the pcre_compile() error code - being within the table. This could lead to an OK return being given in - error. - -2. Changed the call to open a subject file in pcregrep from fopen(pathname, - "r") to fopen(pathname, "rb"), which fixed a problem with some of the tests - in a Windows environment. - -3. The pcregrep --count option prints the count for each file even when it is - zero, as does GNU grep. However, pcregrep was also printing all files when - --files-with-matches was added. Now, when both options are given, it prints - counts only for those files that have at least one match. (GNU grep just - prints the file name in this circumstance, but including the count seems - more useful - otherwise, why use --count?) Also ensured that the - combination -clh just lists non-zero counts, with no names. - -4. The long form of the pcregrep -F option was incorrectly implemented as - --fixed_strings instead of --fixed-strings. This is an incompatible change, - but it seems right to fix it, and I didn't think it was worth preserving - the old behaviour. - -5. The command line items --regex=pattern and --regexp=pattern were not - recognized by pcregrep, which required --regex pattern or --regexp pattern - (with a space rather than an '='). The man page documented the '=' forms, - which are compatible with GNU grep; these now work. - -6. No libpcreposix.pc file was created for pkg-config; there was just - libpcre.pc and libpcrecpp.pc. The omission has been rectified. - -7. Added #ifndef SUPPORT_UCP into the pcre_ucd.c module, to reduce its size - when UCP support is not needed, by modifying the Python script that - generates it from Unicode data files. This should not matter if the module - is correctly used as a library, but I received one complaint about 50K of - unwanted data. My guess is that the person linked everything into his - program rather than using a library. Anyway, it does no harm. - -8. A pattern such as /\x{123}{2,2}+/8 was incorrectly compiled; the trigger - was a minimum greater than 1 for a wide character in a possessive - repetition. The same bug could also affect patterns like /(\x{ff}{0,2})*/8 - which had an unlimited repeat of a nested, fixed maximum repeat of a wide - character. Chaos in the form of incorrect output or a compiling loop could - result. - -9. The restrictions on what a pattern can contain when partial matching is - requested for pcre_exec() have been removed. All patterns can now be - partially matched by this function. In addition, if there are at least two - slots in the offset vector, the offset of the earliest inspected character - for the match and the offset of the end of the subject are set in them when - PCRE_ERROR_PARTIAL is returned. - -10. Partial matching has been split into two forms: PCRE_PARTIAL_SOFT, which is - synonymous with PCRE_PARTIAL, for backwards compatibility, and - PCRE_PARTIAL_HARD, which causes a partial match to supersede a full match, - and may be more useful for multi-segment matching. - -11. Partial matching with pcre_exec() is now more intuitive. A partial match - used to be given if ever the end of the subject was reached; now it is - given only if matching could not proceed because another character was - needed. This makes a difference in some odd cases such as Z(*FAIL) with the - string "Z", which now yields "no match" instead of "partial match". In the - case of pcre_dfa_exec(), "no match" is given if every matching path for the - final character ended with (*FAIL). - -12. Restarting a match using pcre_dfa_exec() after a partial match did not work - if the pattern had a "must contain" character that was already found in the - earlier partial match, unless partial matching was again requested. For - example, with the pattern /dog.(body)?/, the "must contain" character is - "g". If the first part-match was for the string "dog", restarting with - "sbody" failed. This bug has been fixed. - -13. The string returned by pcre_dfa_exec() after a partial match has been - changed so that it starts at the first inspected character rather than the - first character of the match. This makes a difference only if the pattern - starts with a lookbehind assertion or \b or \B (\K is not supported by - pcre_dfa_exec()). It's an incompatible change, but it makes the two - matching functions compatible, and I think it's the right thing to do. - -14. Added a pcredemo man page, created automatically from the pcredemo.c file, - so that the demonstration program is easily available in environments where - PCRE has not been installed from source. - -15. Arranged to add -DPCRE_STATIC to cflags in libpcre.pc, libpcreposix.cp, - libpcrecpp.pc and pcre-config when PCRE is not compiled as a shared - library. - -16. Added REG_UNGREEDY to the pcreposix interface, at the request of a user. - It maps to PCRE_UNGREEDY. It is not, of course, POSIX-compatible, but it - is not the first non-POSIX option to be added. Clearly some people find - these options useful. - -17. If a caller to the POSIX matching function regexec() passes a non-zero - value for nmatch with a NULL value for pmatch, the value of - nmatch is forced to zero. - -18. RunGrepTest did not have a test for the availability of the -u option of - the diff command, as RunTest does. It now checks in the same way as - RunTest, and also checks for the -b option. - -19. If an odd number of negated classes containing just a single character - interposed, within parentheses, between a forward reference to a named - subpattern and the definition of the subpattern, compilation crashed with - an internal error, complaining that it could not find the referenced - subpattern. An example of a crashing pattern is /(?&A)(([^m])(?<A>))/. - [The bug was that it was starting one character too far in when skipping - over the character class, thus treating the ] as data rather than - terminating the class. This meant it could skip too much.] - -20. Added PCRE_NOTEMPTY_ATSTART in order to be able to correctly implement the - /g option in pcretest when the pattern contains \K, which makes it possible - to have an empty string match not at the start, even when the pattern is - anchored. Updated pcretest and pcredemo to use this option. - -21. If the maximum number of capturing subpatterns in a recursion was greater - than the maximum at the outer level, the higher number was returned, but - with unset values at the outer level. The correct (outer level) value is - now given. - -22. If (*ACCEPT) appeared inside capturing parentheses, previous releases of - PCRE did not set those parentheses (unlike Perl). I have now found a way to - make it do so. The string so far is captured, making this feature - compatible with Perl. - -23. The tests have been re-organized, adding tests 11 and 12, to make it - possible to check the Perl 5.10 features against Perl 5.10. - -24. Perl 5.10 allows subroutine calls in lookbehinds, as long as the subroutine - pattern matches a fixed length string. PCRE did not allow this; now it - does. Neither allows recursion. - -25. I finally figured out how to implement a request to provide the minimum - length of subject string that was needed in order to match a given pattern. - (It was back references and recursion that I had previously got hung up - on.) This code has now been added to pcre_study(); it finds a lower bound - to the length of subject needed. It is not necessarily the greatest lower - bound, but using it to avoid searching strings that are too short does give - some useful speed-ups. The value is available to calling programs via - pcre_fullinfo(). - -26. While implementing 25, I discovered to my embarrassment that pcretest had - not been passing the result of pcre_study() to pcre_dfa_exec(), so the - study optimizations had never been tested with that matching function. - Oops. What is worse, even when it was passed study data, there was a bug in - pcre_dfa_exec() that meant it never actually used it. Double oops. There - were also very few tests of studied patterns with pcre_dfa_exec(). - -27. If (?| is used to create subpatterns with duplicate numbers, they are now - allowed to have the same name, even if PCRE_DUPNAMES is not set. However, - on the other side of the coin, they are no longer allowed to have different - names, because these cannot be distinguished in PCRE, and this has caused - confusion. (This is a difference from Perl.) - -28. When duplicate subpattern names are present (necessarily with different - numbers, as required by 27 above), and a test is made by name in a - conditional pattern, either for a subpattern having been matched, or for - recursion in such a pattern, all the associated numbered subpatterns are - tested, and the overall condition is true if the condition is true for any - one of them. This is the way Perl works, and is also more like the way - testing by number works. - - -Version 7.9 11-Apr-09 ---------------------- - -1. When building with support for bzlib/zlib (pcregrep) and/or readline - (pcretest), all targets were linked against these libraries. This included - libpcre, libpcreposix, and libpcrecpp, even though they do not use these - libraries. This caused unwanted dependencies to be created. This problem - has been fixed, and now only pcregrep is linked with bzlib/zlib and only - pcretest is linked with readline. - -2. The "typedef int BOOL" in pcre_internal.h that was included inside the - "#ifndef FALSE" condition by an earlier change (probably 7.8/18) has been - moved outside it again, because FALSE and TRUE are already defined in AIX, - but BOOL is not. - -3. The pcre_config() function was treating the PCRE_MATCH_LIMIT and - PCRE_MATCH_LIMIT_RECURSION values as ints, when they should be long ints. - -4. The pcregrep documentation said spaces were inserted as well as colons (or - hyphens) following file names and line numbers when outputting matching - lines. This is not true; no spaces are inserted. I have also clarified the - wording for the --colour (or --color) option. - -5. In pcregrep, when --colour was used with -o, the list of matching strings - was not coloured; this is different to GNU grep, so I have changed it to be - the same. - -6. When --colo(u)r was used in pcregrep, only the first matching substring in - each matching line was coloured. Now it goes on to look for further matches - of any of the test patterns, which is the same behaviour as GNU grep. - -7. A pattern that could match an empty string could cause pcregrep to loop; it - doesn't make sense to accept an empty string match in pcregrep, so I have - locked it out (using PCRE's PCRE_NOTEMPTY option). By experiment, this - seems to be how GNU grep behaves. [But see later change 40 for release - 8.33.] - -8. The pattern (?(?=.*b)b|^) was incorrectly compiled as "match must be at - start or after a newline", because the conditional assertion was not being - correctly handled. The rule now is that both the assertion and what follows - in the first alternative must satisfy the test. - -9. If auto-callout was enabled in a pattern with a conditional group whose - condition was an assertion, PCRE could crash during matching, both with - pcre_exec() and pcre_dfa_exec(). - -10. The PCRE_DOLLAR_ENDONLY option was not working when pcre_dfa_exec() was - used for matching. - -11. Unicode property support in character classes was not working for - characters (bytes) greater than 127 when not in UTF-8 mode. - -12. Added the -M command line option to pcretest. - -14. Added the non-standard REG_NOTEMPTY option to the POSIX interface. - -15. Added the PCRE_NO_START_OPTIMIZE match-time option. - -16. Added comments and documentation about mis-use of no_arg in the C++ - wrapper. - -17. Implemented support for UTF-8 encoding in EBCDIC environments, a patch - from Martin Jerabek that uses macro names for all relevant character and - string constants. - -18. Added to pcre_internal.h two configuration checks: (a) If both EBCDIC and - SUPPORT_UTF8 are set, give an error; (b) If SUPPORT_UCP is set without - SUPPORT_UTF8, define SUPPORT_UTF8. The "configure" script handles both of - these, but not everybody uses configure. - -19. A conditional group that had only one branch was not being correctly - recognized as an item that could match an empty string. This meant that an - enclosing group might also not be so recognized, causing infinite looping - (and probably a segfault) for patterns such as ^"((?(?=[a])[^"])|b)*"$ - with the subject "ab", where knowledge that the repeated group can match - nothing is needed in order to break the loop. - -20. If a pattern that was compiled with callouts was matched using pcre_dfa_ - exec(), but without supplying a callout function, matching went wrong. - -21. If PCRE_ERROR_MATCHLIMIT occurred during a recursion, there was a memory - leak if the size of the offset vector was greater than 30. When the vector - is smaller, the saved offsets during recursion go onto a local stack - vector, but for larger vectors malloc() is used. It was failing to free - when the recursion yielded PCRE_ERROR_MATCH_LIMIT (or any other "abnormal" - error, in fact). - -22. There was a missing #ifdef SUPPORT_UTF8 round one of the variables in the - heapframe that is used only when UTF-8 support is enabled. This caused no - problem, but was untidy. - -23. Steven Van Ingelgem's patch to CMakeLists.txt to change the name - CMAKE_BINARY_DIR to PROJECT_BINARY_DIR so that it works when PCRE is - included within another project. - -24. Steven Van Ingelgem's patches to add more options to the CMake support, - slightly modified by me: - - (a) PCRE_BUILD_TESTS can be set OFF not to build the tests, including - not building pcregrep. - - (b) PCRE_BUILD_PCREGREP can be see OFF not to build pcregrep, but only - if PCRE_BUILD_TESTS is also set OFF, because the tests use pcregrep. - -25. Forward references, both numeric and by name, in patterns that made use of - duplicate group numbers, could behave incorrectly or give incorrect errors, - because when scanning forward to find the reference group, PCRE was not - taking into account the duplicate group numbers. A pattern such as - ^X(?3)(a)(?|(b)|(q))(Y) is an example. - -26. Changed a few more instances of "const unsigned char *" to USPTR, making - the feature of a custom pointer more persuasive (as requested by a user). - -27. Wrapped the definitions of fileno and isatty for Windows, which appear in - pcretest.c, inside #ifndefs, because it seems they are sometimes already - pre-defined. - -28. Added support for (*UTF8) at the start of a pattern. - -29. Arrange for flags added by the "release type" setting in CMake to be shown - in the configuration summary. - - -Version 7.8 05-Sep-08 ---------------------- - -1. Replaced UCP searching code with optimized version as implemented for Ad - Muncher (http://www.admuncher.com/) by Peter Kankowski. This uses a two- - stage table and inline lookup instead of a function, giving speed ups of 2 - to 5 times on some simple patterns that I tested. Permission was given to - distribute the MultiStage2.py script that generates the tables (it's not in - the tarball, but is in the Subversion repository). - -2. Updated the Unicode datatables to Unicode 5.1.0. This adds yet more - scripts. - -3. Change 12 for 7.7 introduced a bug in pcre_study() when a pattern contained - a group with a zero qualifier. The result of the study could be incorrect, - or the function might crash, depending on the pattern. - -4. Caseless matching was not working for non-ASCII characters in back - references. For example, /(\x{de})\1/8i was not matching \x{de}\x{fe}. - It now works when Unicode Property Support is available. - -5. In pcretest, an escape such as \x{de} in the data was always generating - a UTF-8 string, even in non-UTF-8 mode. Now it generates a single byte in - non-UTF-8 mode. If the value is greater than 255, it gives a warning about - truncation. - -6. Minor bugfix in pcrecpp.cc (change "" == ... to NULL == ...). - -7. Added two (int) casts to pcregrep when printing the difference of two - pointers, in case they are 64-bit values. - -8. Added comments about Mac OS X stack usage to the pcrestack man page and to - test 2 if it fails. - -9. Added PCRE_CALL_CONVENTION just before the names of all exported functions, - and a #define of that name to empty if it is not externally set. This is to - allow users of MSVC to set it if necessary. - -10. The PCRE_EXP_DEFN macro which precedes exported functions was missing from - the convenience functions in the pcre_get.c source file. - -11. An option change at the start of a pattern that had top-level alternatives - could cause overwriting and/or a crash. This command provoked a crash in - some environments: - - printf "/(?i)[\xc3\xa9\xc3\xbd]|[\xc3\xa9\xc3\xbdA]/8\n" | pcretest - - This potential security problem was recorded as CVE-2008-2371. - -12. For a pattern where the match had to start at the beginning or immediately - after a newline (e.g /.*anything/ without the DOTALL flag), pcre_exec() and - pcre_dfa_exec() could read past the end of the passed subject if there was - no match. To help with detecting such bugs (e.g. with valgrind), I modified - pcretest so that it places the subject at the end of its malloc-ed buffer. - -13. The change to pcretest in 12 above threw up a couple more cases when pcre_ - exec() might read past the end of the data buffer in UTF-8 mode. - -14. A similar bug to 7.3/2 existed when the PCRE_FIRSTLINE option was set and - the data contained the byte 0x85 as part of a UTF-8 character within its - first line. This applied both to normal and DFA matching. - -15. Lazy qualifiers were not working in some cases in UTF-8 mode. For example, - /^[^d]*?$/8 failed to match "abc". - -16. Added a missing copyright notice to pcrecpp_internal.h. - -17. Make it more clear in the documentation that values returned from - pcre_exec() in ovector are byte offsets, not character counts. - -18. Tidied a few places to stop certain compilers from issuing warnings. - -19. Updated the Virtual Pascal + BCC files to compile the latest v7.7, as - supplied by Stefan Weber. I made a further small update for 7.8 because - there is a change of source arrangements: the pcre_searchfuncs.c module is - replaced by pcre_ucd.c. - - -Version 7.7 07-May-08 ---------------------- - -1. Applied Craig's patch to sort out a long long problem: "If we can't convert - a string to a long long, pretend we don't even have a long long." This is - done by checking for the strtoq, strtoll, and _strtoi64 functions. - -2. Applied Craig's patch to pcrecpp.cc to restore ABI compatibility with - pre-7.6 versions, which defined a global no_arg variable instead of putting - it in the RE class. (See also #8 below.) - -3. Remove a line of dead code, identified by coverity and reported by Nuno - Lopes. - -4. Fixed two related pcregrep bugs involving -r with --include or --exclude: - - (1) The include/exclude patterns were being applied to the whole pathnames - of files, instead of just to the final components. - - (2) If there was more than one level of directory, the subdirectories were - skipped unless they satisfied the include/exclude conditions. This is - inconsistent with GNU grep (and could even be seen as contrary to the - pcregrep specification - which I improved to make it absolutely clear). - The action now is always to scan all levels of directory, and just - apply the include/exclude patterns to regular files. - -5. Added the --include_dir and --exclude_dir patterns to pcregrep, and used - --exclude_dir in the tests to avoid scanning .svn directories. - -6. Applied Craig's patch to the QuoteMeta function so that it escapes the - NUL character as backslash + 0 rather than backslash + NUL, because PCRE - doesn't support NULs in patterns. - -7. Added some missing "const"s to declarations of static tables in - pcre_compile.c and pcre_dfa_exec.c. - -8. Applied Craig's patch to pcrecpp.cc to fix a problem in OS X that was - caused by fix #2 above. (Subsequently also a second patch to fix the - first patch. And a third patch - this was a messy problem.) - -9. Applied Craig's patch to remove the use of push_back(). - -10. Applied Alan Lehotsky's patch to add REG_STARTEND support to the POSIX - matching function regexec(). - -11. Added support for the Oniguruma syntax \g<name>, \g<n>, \g'name', \g'n', - which, however, unlike Perl's \g{...}, are subroutine calls, not back - references. PCRE supports relative numbers with this syntax (I don't think - Oniguruma does). - -12. Previously, a group with a zero repeat such as (...){0} was completely - omitted from the compiled regex. However, this means that if the group - was called as a subroutine from elsewhere in the pattern, things went wrong - (an internal error was given). Such groups are now left in the compiled - pattern, with a new opcode that causes them to be skipped at execution - time. - -13. Added the PCRE_JAVASCRIPT_COMPAT option. This makes the following changes - to the way PCRE behaves: - - (a) A lone ] character is dis-allowed (Perl treats it as data). - - (b) A back reference to an unmatched subpattern matches an empty string - (Perl fails the current match path). - - (c) A data ] in a character class must be notated as \] because if the - first data character in a class is ], it defines an empty class. (In - Perl it is not possible to have an empty class.) The empty class [] - never matches; it forces failure and is equivalent to (*FAIL) or (?!). - The negative empty class [^] matches any one character, independently - of the DOTALL setting. - -14. A pattern such as /(?2)[]a()b](abc)/ which had a forward reference to a - non-existent subpattern following a character class starting with ']' and - containing () gave an internal compiling error instead of "reference to - non-existent subpattern". Fortunately, when the pattern did exist, the - compiled code was correct. (When scanning forwards to check for the - existence of the subpattern, it was treating the data ']' as terminating - the class, so got the count wrong. When actually compiling, the reference - was subsequently set up correctly.) - -15. The "always fail" assertion (?!) is optimzed to (*FAIL) by pcre_compile; - it was being rejected as not supported by pcre_dfa_exec(), even though - other assertions are supported. I have made pcre_dfa_exec() support - (*FAIL). - -16. The implementation of 13c above involved the invention of a new opcode, - OP_ALLANY, which is like OP_ANY but doesn't check the /s flag. Since /s - cannot be changed at match time, I realized I could make a small - improvement to matching performance by compiling OP_ALLANY instead of - OP_ANY for "." when DOTALL was set, and then removing the runtime tests - on the OP_ANY path. - -17. Compiling pcretest on Windows with readline support failed without the - following two fixes: (1) Make the unistd.h include conditional on - HAVE_UNISTD_H; (2) #define isatty and fileno as _isatty and _fileno. - -18. Changed CMakeLists.txt and cmake/FindReadline.cmake to arrange for the - ncurses library to be included for pcretest when ReadLine support is - requested, but also to allow for it to be overridden. This patch came from - Daniel Bergström. - -19. There was a typo in the file ucpinternal.h where f0_rangeflag was defined - as 0x00f00000 instead of 0x00800000. Luckily, this would not have caused - any errors with the current Unicode tables. Thanks to Peter Kankowski for - spotting this. - - -Version 7.6 28-Jan-08 ---------------------- - -1. A character class containing a very large number of characters with - codepoints greater than 255 (in UTF-8 mode, of course) caused a buffer - overflow. - -2. Patch to cut out the "long long" test in pcrecpp_unittest when - HAVE_LONG_LONG is not defined. - -3. Applied Christian Ehrlicher's patch to update the CMake build files to - bring them up to date and include new features. This patch includes: - - - Fixed PH's badly added libz and libbz2 support. - - Fixed a problem with static linking. - - Added pcredemo. [But later removed - see 7 below.] - - Fixed dftables problem and added an option. - - Added a number of HAVE_XXX tests, including HAVE_WINDOWS_H and - HAVE_LONG_LONG. - - Added readline support for pcretest. - - Added an listing of the option settings after cmake has run. - -4. A user submitted a patch to Makefile that makes it easy to create - "pcre.dll" under mingw when using Configure/Make. I added stuff to - Makefile.am that cause it to include this special target, without - affecting anything else. Note that the same mingw target plus all - the other distribution libraries and programs are now supported - when configuring with CMake (see 6 below) instead of with - Configure/Make. - -5. Applied Craig's patch that moves no_arg into the RE class in the C++ code. - This is an attempt to solve the reported problem "pcrecpp::no_arg is not - exported in the Windows port". It has not yet been confirmed that the patch - solves the problem, but it does no harm. - -6. Applied Sheri's patch to CMakeLists.txt to add NON_STANDARD_LIB_PREFIX and - NON_STANDARD_LIB_SUFFIX for dll names built with mingw when configured - with CMake, and also correct the comment about stack recursion. - -7. Remove the automatic building of pcredemo from the ./configure system and - from CMakeLists.txt. The whole idea of pcredemo.c is that it is an example - of a program that users should build themselves after PCRE is installed, so - building it automatically is not really right. What is more, it gave - trouble in some build environments. - -8. Further tidies to CMakeLists.txt from Sheri and Christian. - - -Version 7.5 10-Jan-08 ---------------------- - -1. Applied a patch from Craig: "This patch makes it possible to 'ignore' - values in parens when parsing an RE using the C++ wrapper." - -2. Negative specials like \S did not work in character classes in UTF-8 mode. - Characters greater than 255 were excluded from the class instead of being - included. - -3. The same bug as (2) above applied to negated POSIX classes such as - [:^space:]. - -4. PCRECPP_STATIC was referenced in pcrecpp_internal.h, but nowhere was it - defined or documented. It seems to have been a typo for PCRE_STATIC, so - I have changed it. - -5. The construct (?&) was not diagnosed as a syntax error (it referenced the - first named subpattern) and a construct such as (?&a) would reference the - first named subpattern whose name started with "a" (in other words, the - length check was missing). Both these problems are fixed. "Subpattern name - expected" is now given for (?&) (a zero-length name), and this patch also - makes it give the same error for \k'' (previously it complained that that - was a reference to a non-existent subpattern). - -6. The erroneous patterns (?+-a) and (?-+a) give different error messages; - this is right because (?- can be followed by option settings as well as by - digits. I have, however, made the messages clearer. - -7. Patterns such as (?(1)a|b) (a pattern that contains fewer subpatterns - than the number used in the conditional) now cause a compile-time error. - This is actually not compatible with Perl, which accepts such patterns, but - treats the conditional as always being FALSE (as PCRE used to), but it - seems to me that giving a diagnostic is better. - -8. Change "alphameric" to the more common word "alphanumeric" in comments - and messages. - -9. Fix two occurrences of "backslash" in comments that should have been - "backspace". - -10. Remove two redundant lines of code that can never be obeyed (their function - was moved elsewhere). - -11. The program that makes PCRE's Unicode character property table had a bug - which caused it to generate incorrect table entries for sequences of - characters that have the same character type, but are in different scripts. - It amalgamated them into a single range, with the script of the first of - them. In other words, some characters were in the wrong script. There were - thirteen such cases, affecting characters in the following ranges: - - U+002b0 - U+002c1 - U+0060c - U+0060d - U+0061e - U+00612 - U+0064b - U+0065e - U+0074d - U+0076d - U+01800 - U+01805 - U+01d00 - U+01d77 - U+01d9b - U+01dbf - U+0200b - U+0200f - U+030fc - U+030fe - U+03260 - U+0327f - U+0fb46 - U+0fbb1 - U+10450 - U+1049d - -12. The -o option (show only the matching part of a line) for pcregrep was not - compatible with GNU grep in that, if there was more than one match in a - line, it showed only the first of them. It now behaves in the same way as - GNU grep. - -13. If the -o and -v options were combined for pcregrep, it printed a blank - line for every non-matching line. GNU grep prints nothing, and pcregrep now - does the same. The return code can be used to tell if there were any - non-matching lines. - -14. Added --file-offsets and --line-offsets to pcregrep. - -15. The pattern (?=something)(?R) was not being diagnosed as a potentially - infinitely looping recursion. The bug was that positive lookaheads were not - being skipped when checking for a possible empty match (negative lookaheads - and both kinds of lookbehind were skipped). - -16. Fixed two typos in the Windows-only code in pcregrep.c, and moved the - inclusion of <windows.h> to before rather than after the definition of - INVALID_FILE_ATTRIBUTES (patch from David Byron). - -17. Specifying a possessive quantifier with a specific limit for a Unicode - character property caused pcre_compile() to compile bad code, which led at - runtime to PCRE_ERROR_INTERNAL (-14). Examples of patterns that caused this - are: /\p{Zl}{2,3}+/8 and /\p{Cc}{2}+/8. It was the possessive "+" that - caused the error; without that there was no problem. - -18. Added --enable-pcregrep-libz and --enable-pcregrep-libbz2. - -19. Added --enable-pcretest-libreadline. - -20. In pcrecpp.cc, the variable 'count' was incremented twice in - RE::GlobalReplace(). As a result, the number of replacements returned was - double what it should be. I removed one of the increments, but Craig sent a - later patch that removed the other one (the right fix) and added unit tests - that check the return values (which was not done before). - -21. Several CMake things: - - (1) Arranged that, when cmake is used on Unix, the libraries end up with - the names libpcre and libpcreposix, not just pcre and pcreposix. - - (2) The above change means that pcretest and pcregrep are now correctly - linked with the newly-built libraries, not previously installed ones. - - (3) Added PCRE_SUPPORT_LIBREADLINE, PCRE_SUPPORT_LIBZ, PCRE_SUPPORT_LIBBZ2. - -22. In UTF-8 mode, with newline set to "any", a pattern such as .*a.*=.b.* - crashed when matching a string such as a\x{2029}b (note that \x{2029} is a - UTF-8 newline character). The key issue is that the pattern starts .*; - this means that the match must be either at the beginning, or after a - newline. The bug was in the code for advancing after a failed match and - checking that the new position followed a newline. It was not taking - account of UTF-8 characters correctly. - -23. PCRE was behaving differently from Perl in the way it recognized POSIX - character classes. PCRE was not treating the sequence [:...:] as a - character class unless the ... were all letters. Perl, however, seems to - allow any characters between [: and :], though of course it rejects as - unknown any "names" that contain non-letters, because all the known class - names consist only of letters. Thus, Perl gives an error for [[:1234:]], - for example, whereas PCRE did not - it did not recognize a POSIX character - class. This seemed a bit dangerous, so the code has been changed to be - closer to Perl. The behaviour is not identical to Perl, because PCRE will - diagnose an unknown class for, for example, [[:l\ower:]] where Perl will - treat it as [[:lower:]]. However, PCRE does now give "unknown" errors where - Perl does, and where it didn't before. - -24. Rewrite so as to remove the single use of %n from pcregrep because in some - Windows environments %n is disabled by default. - - -Version 7.4 21-Sep-07 ---------------------- - -1. Change 7.3/28 was implemented for classes by looking at the bitmap. This - means that a class such as [\s] counted as "explicit reference to CR or - LF". That isn't really right - the whole point of the change was to try to - help when there was an actual mention of one of the two characters. So now - the change happens only if \r or \n (or a literal CR or LF) character is - encountered. - -2. The 32-bit options word was also used for 6 internal flags, but the numbers - of both had grown to the point where there were only 3 bits left. - Fortunately, there was spare space in the data structure, and so I have - moved the internal flags into a new 16-bit field to free up more option - bits. - -3. The appearance of (?J) at the start of a pattern set the DUPNAMES option, - but did not set the internal JCHANGED flag - either of these is enough to - control the way the "get" function works - but the PCRE_INFO_JCHANGED - facility is supposed to tell if (?J) was ever used, so now (?J) at the - start sets both bits. - -4. Added options (at build time, compile time, exec time) to change \R from - matching any Unicode line ending sequence to just matching CR, LF, or CRLF. - -5. doc/pcresyntax.html was missing from the distribution. - -6. Put back the definition of PCRE_ERROR_NULLWSLIMIT, for backward - compatibility, even though it is no longer used. - -7. Added macro for snprintf to pcrecpp_unittest.cc and also for strtoll and - strtoull to pcrecpp.cc to select the available functions in WIN32 when the - windows.h file is present (where different names are used). [This was - reversed later after testing - see 16 below.] - -8. Changed all #include <config.h> to #include "config.h". There were also - some further <pcre.h> cases that I changed to "pcre.h". - -9. When pcregrep was used with the --colour option, it missed the line ending - sequence off the lines that it output. - -10. It was pointed out to me that arrays of string pointers cause lots of - relocations when a shared library is dynamically loaded. A technique of - using a single long string with a table of offsets can drastically reduce - these. I have refactored PCRE in four places to do this. The result is - dramatic: - - Originally: 290 - After changing UCP table: 187 - After changing error message table: 43 - After changing table of "verbs" 36 - After changing table of Posix names 22 - - Thanks to the folks working on Gregex for glib for this insight. - -11. --disable-stack-for-recursion caused compiling to fail unless -enable- - unicode-properties was also set. - -12. Updated the tests so that they work when \R is defaulted to ANYCRLF. - -13. Added checks for ANY and ANYCRLF to pcrecpp.cc where it previously - checked only for CRLF. - -14. Added casts to pcretest.c to avoid compiler warnings. - -15. Added Craig's patch to various pcrecpp modules to avoid compiler warnings. - -16. Added Craig's patch to remove the WINDOWS_H tests, that were not working, - and instead check for _strtoi64 explicitly, and avoid the use of snprintf() - entirely. This removes changes made in 7 above. - -17. The CMake files have been updated, and there is now more information about - building with CMake in the NON-UNIX-USE document. - - -Version 7.3 28-Aug-07 ---------------------- - - 1. In the rejigging of the build system that eventually resulted in 7.1, the - line "#include <pcre.h>" was included in pcre_internal.h. The use of angle - brackets there is not right, since it causes compilers to look for an - installed pcre.h, not the version that is in the source that is being - compiled (which of course may be different). I have changed it back to: - - #include "pcre.h" - - I have a vague recollection that the change was concerned with compiling in - different directories, but in the new build system, that is taken care of - by the VPATH setting the Makefile. - - 2. The pattern .*$ when run in not-DOTALL UTF-8 mode with newline=any failed - when the subject happened to end in the byte 0x85 (e.g. if the last - character was \x{1ec5}). *Character* 0x85 is one of the "any" newline - characters but of course it shouldn't be taken as a newline when it is part - of another character. The bug was that, for an unlimited repeat of . in - not-DOTALL UTF-8 mode, PCRE was advancing by bytes rather than by - characters when looking for a newline. - - 3. A small performance improvement in the DOTALL UTF-8 mode .* case. - - 4. Debugging: adjusted the names of opcodes for different kinds of parentheses - in debug output. - - 5. Arrange to use "%I64d" instead of "%lld" and "%I64u" instead of "%llu" for - long printing in the pcrecpp unittest when running under MinGW. - - 6. ESC_K was left out of the EBCDIC table. - - 7. Change 7.0/38 introduced a new limit on the number of nested non-capturing - parentheses; I made it 1000, which seemed large enough. Unfortunately, the - limit also applies to "virtual nesting" when a pattern is recursive, and in - this case 1000 isn't so big. I have been able to remove this limit at the - expense of backing off one optimization in certain circumstances. Normally, - when pcre_exec() would call its internal match() function recursively and - immediately return the result unconditionally, it uses a "tail recursion" - feature to save stack. However, when a subpattern that can match an empty - string has an unlimited repetition quantifier, it no longer makes this - optimization. That gives it a stack frame in which to save the data for - checking that an empty string has been matched. Previously this was taken - from the 1000-entry workspace that had been reserved. So now there is no - explicit limit, but more stack is used. - - 8. Applied Daniel's patches to solve problems with the import/export magic - syntax that is required for Windows, and which was going wrong for the - pcreposix and pcrecpp parts of the library. These were overlooked when this - problem was solved for the main library. - - 9. There were some crude static tests to avoid integer overflow when computing - the size of patterns that contain repeated groups with explicit upper - limits. As the maximum quantifier is 65535, the maximum group length was - set at 30,000 so that the product of these two numbers did not overflow a - 32-bit integer. However, it turns out that people want to use groups that - are longer than 30,000 bytes (though not repeat them that many times). - Change 7.0/17 (the refactoring of the way the pattern size is computed) has - made it possible to implement the integer overflow checks in a much more - dynamic way, which I have now done. The artificial limitation on group - length has been removed - we now have only the limit on the total length of - the compiled pattern, which depends on the LINK_SIZE setting. - -10. Fixed a bug in the documentation for get/copy named substring when - duplicate names are permitted. If none of the named substrings are set, the - functions return PCRE_ERROR_NOSUBSTRING (7); the doc said they returned an - empty string. - -11. Because Perl interprets \Q...\E at a high level, and ignores orphan \E - instances, patterns such as [\Q\E] or [\E] or even [^\E] cause an error, - because the ] is interpreted as the first data character and the - terminating ] is not found. PCRE has been made compatible with Perl in this - regard. Previously, it interpreted [\Q\E] as an empty class, and [\E] could - cause memory overwriting. - -10. Like Perl, PCRE automatically breaks an unlimited repeat after an empty - string has been matched (to stop an infinite loop). It was not recognizing - a conditional subpattern that could match an empty string if that - subpattern was within another subpattern. For example, it looped when - trying to match (((?(1)X|))*) but it was OK with ((?(1)X|)*) where the - condition was not nested. This bug has been fixed. - -12. A pattern like \X?\d or \P{L}?\d in non-UTF-8 mode could cause a backtrack - past the start of the subject in the presence of bytes with the top bit - set, for example "\x8aBCD". - -13. Added Perl 5.10 experimental backtracking controls (*FAIL), (*F), (*PRUNE), - (*SKIP), (*THEN), (*COMMIT), and (*ACCEPT). - -14. Optimized (?!) to (*FAIL). - -15. Updated the test for a valid UTF-8 string to conform to the later RFC 3629. - This restricts code points to be within the range 0 to 0x10FFFF, excluding - the "low surrogate" sequence 0xD800 to 0xDFFF. Previously, PCRE allowed the - full range 0 to 0x7FFFFFFF, as defined by RFC 2279. Internally, it still - does: it's just the validity check that is more restrictive. - -16. Inserted checks for integer overflows during escape sequence (backslash) - processing, and also fixed erroneous offset values for syntax errors during - backslash processing. - -17. Fixed another case of looking too far back in non-UTF-8 mode (cf 12 above) - for patterns like [\PPP\x8a]{1,}\x80 with the subject "A\x80". - -18. An unterminated class in a pattern like (?1)\c[ with a "forward reference" - caused an overrun. - -19. A pattern like (?:[\PPa*]*){8,} which had an "extended class" (one with - something other than just ASCII characters) inside a group that had an - unlimited repeat caused a loop at compile time (while checking to see - whether the group could match an empty string). - -20. Debugging a pattern containing \p or \P could cause a crash. For example, - [\P{Any}] did so. (Error in the code for printing property names.) - -21. An orphan \E inside a character class could cause a crash. - -22. A repeated capturing bracket such as (A)? could cause a wild memory - reference during compilation. - -23. There are several functions in pcre_compile() that scan along a compiled - expression for various reasons (e.g. to see if it's fixed length for look - behind). There were bugs in these functions when a repeated \p or \P was - present in the pattern. These operators have additional parameters compared - with \d, etc, and these were not being taken into account when moving along - the compiled data. Specifically: - - (a) A item such as \p{Yi}{3} in a lookbehind was not treated as fixed - length. - - (b) An item such as \pL+ within a repeated group could cause crashes or - loops. - - (c) A pattern such as \p{Yi}+(\P{Yi}+)(?1) could give an incorrect - "reference to non-existent subpattern" error. - - (d) A pattern like (\P{Yi}{2}\277)? could loop at compile time. - -24. A repeated \S or \W in UTF-8 mode could give wrong answers when multibyte - characters were involved (for example /\S{2}/8g with "A\x{a3}BC"). - -25. Using pcregrep in multiline, inverted mode (-Mv) caused it to loop. - -26. Patterns such as [\P{Yi}A] which include \p or \P and just one other - character were causing crashes (broken optimization). - -27. Patterns such as (\P{Yi}*\277)* (group with possible zero repeat containing - \p or \P) caused a compile-time loop. - -28. More problems have arisen in unanchored patterns when CRLF is a valid line - break. For example, the unstudied pattern [\r\n]A does not match the string - "\r\nA" because change 7.0/46 below moves the current point on by two - characters after failing to match at the start. However, the pattern \nA - *does* match, because it doesn't start till \n, and if [\r\n]A is studied, - the same is true. There doesn't seem any very clean way out of this, but - what I have chosen to do makes the common cases work: PCRE now takes note - of whether there can be an explicit match for \r or \n anywhere in the - pattern, and if so, 7.0/46 no longer applies. As part of this change, - there's a new PCRE_INFO_HASCRORLF option for finding out whether a compiled - pattern has explicit CR or LF references. - -29. Added (*CR) etc for changing newline setting at start of pattern. - - -Version 7.2 19-Jun-07 ---------------------- - - 1. If the fr_FR locale cannot be found for test 3, try the "french" locale, - which is apparently normally available under Windows. - - 2. Re-jig the pcregrep tests with different newline settings in an attempt - to make them independent of the local environment's newline setting. - - 3. Add code to configure.ac to remove -g from the CFLAGS default settings. - - 4. Some of the "internals" tests were previously cut out when the link size - was not 2, because the output contained actual offsets. The recent new - "Z" feature of pcretest means that these can be cut out, making the tests - usable with all link sizes. - - 5. Implemented Stan Switzer's goto replacement for longjmp() when not using - stack recursion. This gives a massive performance boost under BSD, but just - a small improvement under Linux. However, it saves one field in the frame - in all cases. - - 6. Added more features from the forthcoming Perl 5.10: - - (a) (?-n) (where n is a string of digits) is a relative subroutine or - recursion call. It refers to the nth most recently opened parentheses. - - (b) (?+n) is also a relative subroutine call; it refers to the nth next - to be opened parentheses. - - (c) Conditions that refer to capturing parentheses can be specified - relatively, for example, (?(-2)... or (?(+3)... - - (d) \K resets the start of the current match so that everything before - is not part of it. - - (e) \k{name} is synonymous with \k<name> and \k'name' (.NET compatible). - - (f) \g{name} is another synonym - part of Perl 5.10's unification of - reference syntax. - - (g) (?| introduces a group in which the numbering of parentheses in each - alternative starts with the same number. - - (h) \h, \H, \v, and \V match horizontal and vertical whitespace. - - 7. Added two new calls to pcre_fullinfo(): PCRE_INFO_OKPARTIAL and - PCRE_INFO_JCHANGED. - - 8. A pattern such as (.*(.)?)* caused pcre_exec() to fail by either not - terminating or by crashing. Diagnosed by Viktor Griph; it was in the code - for detecting groups that can match an empty string. - - 9. A pattern with a very large number of alternatives (more than several - hundred) was running out of internal workspace during the pre-compile - phase, where pcre_compile() figures out how much memory will be needed. A - bit of new cunning has reduced the workspace needed for groups with - alternatives. The 1000-alternative test pattern now uses 12 bytes of - workspace instead of running out of the 4096 that are available. - -10. Inserted some missing (unsigned int) casts to get rid of compiler warnings. - -11. Applied patch from Google to remove an optimization that didn't quite work. - The report of the bug said: - - pcrecpp::RE("a*").FullMatch("aaa") matches, while - pcrecpp::RE("a*?").FullMatch("aaa") does not, and - pcrecpp::RE("a*?\\z").FullMatch("aaa") does again. - -12. If \p or \P was used in non-UTF-8 mode on a character greater than 127 - it matched the wrong number of bytes. - - -Version 7.1 24-Apr-07 ---------------------- - - 1. Applied Bob Rossi and Daniel G's patches to convert the build system to one - that is more "standard", making use of automake and other Autotools. There - is some re-arrangement of the files and adjustment of comments consequent - on this. - - 2. Part of the patch fixed a problem with the pcregrep tests. The test of -r - for recursive directory scanning broke on some systems because the files - are not scanned in any specific order and on different systems the order - was different. A call to "sort" has been inserted into RunGrepTest for the - approprate test as a short-term fix. In the longer term there may be an - alternative. - - 3. I had an email from Eric Raymond about problems translating some of PCRE's - man pages to HTML (despite the fact that I distribute HTML pages, some - people do their own conversions for various reasons). The problems - concerned the use of low-level troff macros .br and .in. I have therefore - removed all such uses from the man pages (some were redundant, some could - be replaced by .nf/.fi pairs). The 132html script that I use to generate - HTML has been updated to handle .nf/.fi and to complain if it encounters - .br or .in. - - 4. Updated comments in configure.ac that get placed in config.h.in and also - arranged for config.h to be included in the distribution, with the name - config.h.generic, for the benefit of those who have to compile without - Autotools (compare pcre.h, which is now distributed as pcre.h.generic). - - 5. Updated the support (such as it is) for Virtual Pascal, thanks to Stefan - Weber: (1) pcre_internal.h was missing some function renames; (2) updated - makevp.bat for the current PCRE, using the additional files - makevp_c.txt, makevp_l.txt, and pcregexp.pas. - - 6. A Windows user reported a minor discrepancy with test 2, which turned out - to be caused by a trailing space on an input line that had got lost in his - copy. The trailing space was an accident, so I've just removed it. - - 7. Add -Wl,-R... flags in pcre-config.in for *BSD* systems, as I'm told - that is needed. - - 8. Mark ucp_table (in ucptable.h) and ucp_gentype (in pcre_ucp_searchfuncs.c) - as "const" (a) because they are and (b) because it helps the PHP - maintainers who have recently made a script to detect big data structures - in the php code that should be moved to the .rodata section. I remembered - to update Builducptable as well, so it won't revert if ucptable.h is ever - re-created. - - 9. Added some extra #ifdef SUPPORT_UTF8 conditionals into pcretest.c, - pcre_printint.src, pcre_compile.c, pcre_study.c, and pcre_tables.c, in - order to be able to cut out the UTF-8 tables in the latter when UTF-8 - support is not required. This saves 1.5-2K of code, which is important in - some applications. - - Later: more #ifdefs are needed in pcre_ord2utf8.c and pcre_valid_utf8.c - so as not to refer to the tables, even though these functions will never be - called when UTF-8 support is disabled. Otherwise there are problems with a - shared library. - -10. Fixed two bugs in the emulated memmove() function in pcre_internal.h: - - (a) It was defining its arguments as char * instead of void *. - - (b) It was assuming that all moves were upwards in memory; this was true - a long time ago when I wrote it, but is no longer the case. - - The emulated memove() is provided for those environments that have neither - memmove() nor bcopy(). I didn't think anyone used it these days, but that - is clearly not the case, as these two bugs were recently reported. - -11. The script PrepareRelease is now distributed: it calls 132html, CleanTxt, - and Detrail to create the HTML documentation, the .txt form of the man - pages, and it removes trailing spaces from listed files. It also creates - pcre.h.generic and config.h.generic from pcre.h and config.h. In the latter - case, it wraps all the #defines with #ifndefs. This script should be run - before "make dist". - -12. Fixed two fairly obscure bugs concerned with quantified caseless matching - with Unicode property support. - - (a) For a maximizing quantifier, if the two different cases of the - character were of different lengths in their UTF-8 codings (there are - some cases like this - I found 11), and the matching function had to - back up over a mixture of the two cases, it incorrectly assumed they - were both the same length. - - (b) When PCRE was configured to use the heap rather than the stack for - recursion during matching, it was not correctly preserving the data for - the other case of a UTF-8 character when checking ahead for a match - while processing a minimizing repeat. If the check also involved - matching a wide character, but failed, corruption could cause an - erroneous result when trying to check for a repeat of the original - character. - -13. Some tidying changes to the testing mechanism: - - (a) The RunTest script now detects the internal link size and whether there - is UTF-8 and UCP support by running ./pcretest -C instead of relying on - values substituted by "configure". (The RunGrepTest script already did - this for UTF-8.) The configure.ac script no longer substitutes the - relevant variables. - - (b) The debugging options /B and /D in pcretest show the compiled bytecode - with length and offset values. This means that the output is different - for different internal link sizes. Test 2 is skipped for link sizes - other than 2 because of this, bypassing the problem. Unfortunately, - there was also a test in test 3 (the locale tests) that used /B and - failed for link sizes other than 2. Rather than cut the whole test out, - I have added a new /Z option to pcretest that replaces the length and - offset values with spaces. This is now used to make test 3 independent - of link size. (Test 2 will be tidied up later.) - -14. If erroroffset was passed as NULL to pcre_compile, it provoked a - segmentation fault instead of returning the appropriate error message. - -15. In multiline mode when the newline sequence was set to "any", the pattern - ^$ would give a match between the \r and \n of a subject such as "A\r\nB". - This doesn't seem right; it now treats the CRLF combination as the line - ending, and so does not match in that case. It's only a pattern such as ^$ - that would hit this one: something like ^ABC$ would have failed after \r - and then tried again after \r\n. - -16. Changed the comparison command for RunGrepTest from "diff -u" to "diff -ub" - in an attempt to make files that differ only in their line terminators - compare equal. This works on Linux. - -17. Under certain error circumstances pcregrep might try to free random memory - as it exited. This is now fixed, thanks to valgrind. - -19. In pcretest, if the pattern /(?m)^$/g<any> was matched against the string - "abc\r\n\r\n", it found an unwanted second match after the second \r. This - was because its rules for how to advance for /g after matching an empty - string at the end of a line did not allow for this case. They now check for - it specially. - -20. pcretest is supposed to handle patterns and data of any length, by - extending its buffers when necessary. It was getting this wrong when the - buffer for a data line had to be extended. - -21. Added PCRE_NEWLINE_ANYCRLF which is like ANY, but matches only CR, LF, or - CRLF as a newline sequence. - -22. Code for handling Unicode properties in pcre_dfa_exec() wasn't being cut - out by #ifdef SUPPORT_UCP. This did no harm, as it could never be used, but - I have nevertheless tidied it up. - -23. Added some casts to kill warnings from HP-UX ia64 compiler. - -24. Added a man page for pcre-config. - - -Version 7.0 19-Dec-06 ---------------------- - - 1. Fixed a signed/unsigned compiler warning in pcre_compile.c, shown up by - moving to gcc 4.1.1. - - 2. The -S option for pcretest uses setrlimit(); I had omitted to #include - sys/time.h, which is documented as needed for this function. It doesn't - seem to matter on Linux, but it showed up on some releases of OS X. - - 3. It seems that there are systems where bytes whose values are greater than - 127 match isprint() in the "C" locale. The "C" locale should be the - default when a C program starts up. In most systems, only ASCII printing - characters match isprint(). This difference caused the output from pcretest - to vary, making some of the tests fail. I have changed pcretest so that: - - (a) When it is outputting text in the compiled version of a pattern, bytes - other than 32-126 are always shown as hex escapes. - - (b) When it is outputting text that is a matched part of a subject string, - it does the same, unless a different locale has been set for the match - (using the /L modifier). In this case, it uses isprint() to decide. - - 4. Fixed a major bug that caused incorrect computation of the amount of memory - required for a compiled pattern when options that changed within the - pattern affected the logic of the preliminary scan that determines the - length. The relevant options are -x, and -i in UTF-8 mode. The result was - that the computed length was too small. The symptoms of this bug were - either the PCRE error "internal error: code overflow" from pcre_compile(), - or a glibc crash with a message such as "pcretest: free(): invalid next - size (fast)". Examples of patterns that provoked this bug (shown in - pcretest format) are: - - /(?-x: )/x - /(?x)(?-x: \s*#\s*)/ - /((?i)[\x{c0}])/8 - /(?i:[\x{c0}])/8 - - HOWEVER: Change 17 below makes this fix obsolete as the memory computation - is now done differently. - - 5. Applied patches from Google to: (a) add a QuoteMeta function to the C++ - wrapper classes; (b) implement a new function in the C++ scanner that is - more efficient than the old way of doing things because it avoids levels of - recursion in the regex matching; (c) add a paragraph to the documentation - for the FullMatch() function. - - 6. The escape sequence \n was being treated as whatever was defined as - "newline". Not only was this contrary to the documentation, which states - that \n is character 10 (hex 0A), but it also went horribly wrong when - "newline" was defined as CRLF. This has been fixed. - - 7. In pcre_dfa_exec.c the value of an unsigned integer (the variable called c) - was being set to -1 for the "end of line" case (supposedly a value that no - character can have). Though this value is never used (the check for end of - line is "zero bytes in current character"), it caused compiler complaints. - I've changed it to 0xffffffff. - - 8. In pcre_version.c, the version string was being built by a sequence of - C macros that, in the event of PCRE_PRERELEASE being defined as an empty - string (as it is for production releases) called a macro with an empty - argument. The C standard says the result of this is undefined. The gcc - compiler treats it as an empty string (which was what was wanted) but it is - reported that Visual C gives an error. The source has been hacked around to - avoid this problem. - - 9. On the advice of a Windows user, included <io.h> and <fcntl.h> in Windows - builds of pcretest, and changed the call to _setmode() to use _O_BINARY - instead of 0x8000. Made all the #ifdefs test both _WIN32 and WIN32 (not all - of them did). - -10. Originally, pcretest opened its input and output without "b"; then I was - told that "b" was needed in some environments, so it was added for release - 5.0 to both the input and output. (It makes no difference on Unix-like - systems.) Later I was told that it is wrong for the input on Windows. I've - now abstracted the modes into two macros, to make it easier to fiddle with - them, and removed "b" from the input mode under Windows. - -11. Added pkgconfig support for the C++ wrapper library, libpcrecpp. - -12. Added -help and --help to pcretest as an official way of being reminded - of the options. - -13. Removed some redundant semicolons after macro calls in pcrecpparg.h.in - and pcrecpp.cc because they annoy compilers at high warning levels. - -14. A bit of tidying/refactoring in pcre_exec.c in the main bumpalong loop. - -15. Fixed an occurrence of == in configure.ac that should have been = (shell - scripts are not C programs :-) and which was not noticed because it works - on Linux. - -16. pcretest is supposed to handle any length of pattern and data line (as one - line or as a continued sequence of lines) by extending its input buffer if - necessary. This feature was broken for very long pattern lines, leading to - a string of junk being passed to pcre_compile() if the pattern was longer - than about 50K. - -17. I have done a major re-factoring of the way pcre_compile() computes the - amount of memory needed for a compiled pattern. Previously, there was code - that made a preliminary scan of the pattern in order to do this. That was - OK when PCRE was new, but as the facilities have expanded, it has become - harder and harder to keep it in step with the real compile phase, and there - have been a number of bugs (see for example, 4 above). I have now found a - cunning way of running the real compile function in a "fake" mode that - enables it to compute how much memory it would need, while actually only - ever using a few hundred bytes of working memory and without too many - tests of the mode. This should make future maintenance and development - easier. A side effect of this work is that the limit of 200 on the nesting - depth of parentheses has been removed (though this was never a serious - limitation, I suspect). However, there is a downside: pcre_compile() now - runs more slowly than before (30% or more, depending on the pattern). I - hope this isn't a big issue. There is no effect on runtime performance. - -18. Fixed a minor bug in pcretest: if a pattern line was not terminated by a - newline (only possible for the last line of a file) and it was a - pattern that set a locale (followed by /Lsomething), pcretest crashed. - -19. Added additional timing features to pcretest. (1) The -tm option now times - matching only, not compiling. (2) Both -t and -tm can be followed, as a - separate command line item, by a number that specifies the number of - repeats to use when timing. The default is 50000; this gives better - precision, but takes uncomfortably long for very large patterns. - -20. Extended pcre_study() to be more clever in cases where a branch of a - subpattern has no definite first character. For example, (a*|b*)[cd] would - previously give no result from pcre_study(). Now it recognizes that the - first character must be a, b, c, or d. - -21. There was an incorrect error "recursive call could loop indefinitely" if - a subpattern (or the entire pattern) that was being tested for matching an - empty string contained only one non-empty item after a nested subpattern. - For example, the pattern (?>\x{100}*)\d(?R) provoked this error - incorrectly, because the \d was being skipped in the check. - -22. The pcretest program now has a new pattern option /B and a command line - option -b, which is equivalent to adding /B to every pattern. This causes - it to show the compiled bytecode, without the additional information that - -d shows. The effect of -d is now the same as -b with -i (and similarly, /D - is the same as /B/I). - -23. A new optimization is now able automatically to treat some sequences such - as a*b as a*+b. More specifically, if something simple (such as a character - or a simple class like \d) has an unlimited quantifier, and is followed by - something that cannot possibly match the quantified thing, the quantifier - is automatically "possessified". - -24. A recursive reference to a subpattern whose number was greater than 39 - went wrong under certain circumstances in UTF-8 mode. This bug could also - have affected the operation of pcre_study(). - -25. Realized that a little bit of performance could be had by replacing - (c & 0xc0) == 0xc0 with c >= 0xc0 when processing UTF-8 characters. - -26. Timing data from pcretest is now shown to 4 decimal places instead of 3. - -27. Possessive quantifiers such as a++ were previously implemented by turning - them into atomic groups such as ($>a+). Now they have their own opcodes, - which improves performance. This includes the automatically created ones - from 23 above. - -28. A pattern such as (?=(\w+))\1: which simulates an atomic group using a - lookahead was broken if it was not anchored. PCRE was mistakenly expecting - the first matched character to be a colon. This applied both to named and - numbered groups. - -29. The ucpinternal.h header file was missing its idempotency #ifdef. - -30. I was sent a "project" file called libpcre.a.dev which I understand makes - building PCRE on Windows easier, so I have included it in the distribution. - -31. There is now a check in pcretest against a ridiculously large number being - returned by pcre_exec() or pcre_dfa_exec(). If this happens in a /g or /G - loop, the loop is abandoned. - -32. Forward references to subpatterns in conditions such as (?(2)...) where - subpattern 2 is defined later cause pcre_compile() to search forwards in - the pattern for the relevant set of parentheses. This search went wrong - when there were unescaped parentheses in a character class, parentheses - escaped with \Q...\E, or parentheses in a #-comment in /x mode. - -33. "Subroutine" calls and backreferences were previously restricted to - referencing subpatterns earlier in the regex. This restriction has now - been removed. - -34. Added a number of extra features that are going to be in Perl 5.10. On the - whole, these are just syntactic alternatives for features that PCRE had - previously implemented using the Python syntax or my own invention. The - other formats are all retained for compatibility. - - (a) Named groups can now be defined as (?<name>...) or (?'name'...) as well - as (?P<name>...). The new forms, as well as being in Perl 5.10, are - also .NET compatible. - - (b) A recursion or subroutine call to a named group can now be defined as - (?&name) as well as (?P>name). - - (c) A backreference to a named group can now be defined as \k<name> or - \k'name' as well as (?P=name). The new forms, as well as being in Perl - 5.10, are also .NET compatible. - - (d) A conditional reference to a named group can now use the syntax - (?(<name>) or (?('name') as well as (?(name). - - (e) A "conditional group" of the form (?(DEFINE)...) can be used to define - groups (named and numbered) that are never evaluated inline, but can be - called as "subroutines" from elsewhere. In effect, the DEFINE condition - is always false. There may be only one alternative in such a group. - - (f) A test for recursion can be given as (?(R1).. or (?(R&name)... as well - as the simple (?(R). The condition is true only if the most recent - recursion is that of the given number or name. It does not search out - through the entire recursion stack. - - (g) The escape \gN or \g{N} has been added, where N is a positive or - negative number, specifying an absolute or relative reference. - -35. Tidied to get rid of some further signed/unsigned compiler warnings and - some "unreachable code" warnings. - -36. Updated the Unicode property tables to Unicode version 5.0.0. Amongst other - things, this adds five new scripts. - -37. Perl ignores orphaned \E escapes completely. PCRE now does the same. - There were also incompatibilities regarding the handling of \Q..\E inside - character classes, for example with patterns like [\Qa\E-\Qz\E] where the - hyphen was adjacent to \Q or \E. I hope I've cleared all this up now. - -38. Like Perl, PCRE detects when an indefinitely repeated parenthesized group - matches an empty string, and forcibly breaks the loop. There were bugs in - this code in non-simple cases. For a pattern such as ^(a()*)* matched - against aaaa the result was just "a" rather than "aaaa", for example. Two - separate and independent bugs (that affected different cases) have been - fixed. - -39. Refactored the code to abolish the use of different opcodes for small - capturing bracket numbers. This is a tidy that I avoided doing when I - removed the limit on the number of capturing brackets for 3.5 back in 2001. - The new approach is not only tidier, it makes it possible to reduce the - memory needed to fix the previous bug (38). - -40. Implemented PCRE_NEWLINE_ANY to recognize any of the Unicode newline - sequences (http://unicode.org/unicode/reports/tr18/) as "newline" when - processing dot, circumflex, or dollar metacharacters, or #-comments in /x - mode. - -41. Add \R to match any Unicode newline sequence, as suggested in the Unicode - report. - -42. Applied patch, originally from Ari Pollak, modified by Google, to allow - copy construction and assignment in the C++ wrapper. - -43. Updated pcregrep to support "--newline=any". In the process, I fixed a - couple of bugs that could have given wrong results in the "--newline=crlf" - case. - -44. Added a number of casts and did some reorganization of signed/unsigned int - variables following suggestions from Dair Grant. Also renamed the variable - "this" as "item" because it is a C++ keyword. - -45. Arranged for dftables to add - - #include "pcre_internal.h" - - to pcre_chartables.c because without it, gcc 4.x may remove the array - definition from the final binary if PCRE is built into a static library and - dead code stripping is activated. - -46. For an unanchored pattern, if a match attempt fails at the start of a - newline sequence, and the newline setting is CRLF or ANY, and the next two - characters are CRLF, advance by two characters instead of one. - - -Version 6.7 04-Jul-06 ---------------------- - - 1. In order to handle tests when input lines are enormously long, pcretest has - been re-factored so that it automatically extends its buffers when - necessary. The code is crude, but this _is_ just a test program. The - default size has been increased from 32K to 50K. - - 2. The code in pcre_study() was using the value of the re argument before - testing it for NULL. (Of course, in any sensible call of the function, it - won't be NULL.) - - 3. The memmove() emulation function in pcre_internal.h, which is used on - systems that lack both memmove() and bcopy() - that is, hardly ever - - was missing a "static" storage class specifier. - - 4. When UTF-8 mode was not set, PCRE looped when compiling certain patterns - containing an extended class (one that cannot be represented by a bitmap - because it contains high-valued characters or Unicode property items, e.g. - [\pZ]). Almost always one would set UTF-8 mode when processing such a - pattern, but PCRE should not loop if you do not (it no longer does). - [Detail: two cases were found: (a) a repeated subpattern containing an - extended class; (b) a recursive reference to a subpattern that followed a - previous extended class. It wasn't skipping over the extended class - correctly when UTF-8 mode was not set.] - - 5. A negated single-character class was not being recognized as fixed-length - in lookbehind assertions such as (?<=[^f]), leading to an incorrect - compile error "lookbehind assertion is not fixed length". - - 6. The RunPerlTest auxiliary script was showing an unexpected difference - between PCRE and Perl for UTF-8 tests. It turns out that it is hard to - write a Perl script that can interpret lines of an input file either as - byte characters or as UTF-8, which is what "perltest" was being required to - do for the non-UTF-8 and UTF-8 tests, respectively. Essentially what you - can't do is switch easily at run time between having the "use utf8;" pragma - or not. In the end, I fudged it by using the RunPerlTest script to insert - "use utf8;" explicitly for the UTF-8 tests. - - 7. In multiline (/m) mode, PCRE was matching ^ after a terminating newline at - the end of the subject string, contrary to the documentation and to what - Perl does. This was true of both matching functions. Now it matches only at - the start of the subject and immediately after *internal* newlines. - - 8. A call of pcre_fullinfo() from pcretest to get the option bits was passing - a pointer to an int instead of a pointer to an unsigned long int. This - caused problems on 64-bit systems. - - 9. Applied a patch from the folks at Google to pcrecpp.cc, to fix "another - instance of the 'standard' template library not being so standard". - -10. There was no check on the number of named subpatterns nor the maximum - length of a subpattern name. The product of these values is used to compute - the size of the memory block for a compiled pattern. By supplying a very - long subpattern name and a large number of named subpatterns, the size - computation could be caused to overflow. This is now prevented by limiting - the length of names to 32 characters, and the number of named subpatterns - to 10,000. - -11. Subpatterns that are repeated with specific counts have to be replicated in - the compiled pattern. The size of memory for this was computed from the - length of the subpattern and the repeat count. The latter is limited to - 65535, but there was no limit on the former, meaning that integer overflow - could in principle occur. The compiled length of a repeated subpattern is - now limited to 30,000 bytes in order to prevent this. - -12. Added the optional facility to have named substrings with the same name. - -13. Added the ability to use a named substring as a condition, using the - Python syntax: (?(name)yes|no). This overloads (?(R)... and names that - are numbers (not recommended). Forward references are permitted. - -14. Added forward references in named backreferences (if you see what I mean). - -15. In UTF-8 mode, with the PCRE_DOTALL option set, a quantified dot in the - pattern could run off the end of the subject. For example, the pattern - "(?s)(.{1,5})"8 did this with the subject "ab". - -16. If PCRE_DOTALL or PCRE_MULTILINE were set, pcre_dfa_exec() behaved as if - PCRE_CASELESS was set when matching characters that were quantified with ? - or *. - -17. A character class other than a single negated character that had a minimum - but no maximum quantifier - for example [ab]{6,} - was not handled - correctly by pce_dfa_exec(). It would match only one character. - -18. A valid (though odd) pattern that looked like a POSIX character - class but used an invalid character after [ (for example [[,abc,]]) caused - pcre_compile() to give the error "Failed: internal error: code overflow" or - in some cases to crash with a glibc free() error. This could even happen if - the pattern terminated after [[ but there just happened to be a sequence of - letters, a binary zero, and a closing ] in the memory that followed. - -19. Perl's treatment of octal escapes in the range \400 to \777 has changed - over the years. Originally (before any Unicode support), just the bottom 8 - bits were taken. Thus, for example, \500 really meant \100. Nowadays the - output from "man perlunicode" includes this: - - The regular expression compiler produces polymorphic opcodes. That - is, the pattern adapts to the data and automatically switches to - the Unicode character scheme when presented with Unicode data--or - instead uses a traditional byte scheme when presented with byte - data. - - Sadly, a wide octal escape does not cause a switch, and in a string with - no other multibyte characters, these octal escapes are treated as before. - Thus, in Perl, the pattern /\500/ actually matches \100 but the pattern - /\500|\x{1ff}/ matches \500 or \777 because the whole thing is treated as a - Unicode string. - - I have not perpetrated such confusion in PCRE. Up till now, it took just - the bottom 8 bits, as in old Perl. I have now made octal escapes with - values greater than \377 illegal in non-UTF-8 mode. In UTF-8 mode they - translate to the appropriate multibyte character. - -29. Applied some refactoring to reduce the number of warnings from Microsoft - and Borland compilers. This has included removing the fudge introduced - seven years ago for the OS/2 compiler (see 2.02/2 below) because it caused - a warning about an unused variable. - -21. PCRE has not included VT (character 0x0b) in the set of whitespace - characters since release 4.0, because Perl (from release 5.004) does not. - [Or at least, is documented not to: some releases seem to be in conflict - with the documentation.] However, when a pattern was studied with - pcre_study() and all its branches started with \s, PCRE still included VT - as a possible starting character. Of course, this did no harm; it just - caused an unnecessary match attempt. - -22. Removed a now-redundant internal flag bit that recorded the fact that case - dependency changed within the pattern. This was once needed for "required - byte" processing, but is no longer used. This recovers a now-scarce options - bit. Also moved the least significant internal flag bit to the most- - significant bit of the word, which was not previously used (hangover from - the days when it was an int rather than a uint) to free up another bit for - the future. - -23. Added support for CRLF line endings as well as CR and LF. As well as the - default being selectable at build time, it can now be changed at runtime - via the PCRE_NEWLINE_xxx flags. There are now options for pcregrep to - specify that it is scanning data with non-default line endings. - -24. Changed the definition of CXXLINK to make it agree with the definition of - LINK in the Makefile, by replacing LDFLAGS to CXXFLAGS. - -25. Applied Ian Taylor's patches to avoid using another stack frame for tail - recursions. This makes a big different to stack usage for some patterns. - -26. If a subpattern containing a named recursion or subroutine reference such - as (?P>B) was quantified, for example (xxx(?P>B)){3}, the calculation of - the space required for the compiled pattern went wrong and gave too small a - value. Depending on the environment, this could lead to "Failed: internal - error: code overflow at offset 49" or "glibc detected double free or - corruption" errors. - -27. Applied patches from Google (a) to support the new newline modes and (b) to - advance over multibyte UTF-8 characters in GlobalReplace. - -28. Change free() to pcre_free() in pcredemo.c. Apparently this makes a - difference for some implementation of PCRE in some Windows version. - -29. Added some extra testing facilities to pcretest: - - \q<number> in a data line sets the "match limit" value - \Q<number> in a data line sets the "match recursion limt" value - -S <number> sets the stack size, where <number> is in megabytes - - The -S option isn't available for Windows. - - -Version 6.6 06-Feb-06 ---------------------- - - 1. Change 16(a) for 6.5 broke things, because PCRE_DATA_SCOPE was not defined - in pcreposix.h. I have copied the definition from pcre.h. - - 2. Change 25 for 6.5 broke compilation in a build directory out-of-tree - because pcre.h is no longer a built file. - - 3. Added Jeff Friedl's additional debugging patches to pcregrep. These are - not normally included in the compiled code. - - -Version 6.5 01-Feb-06 ---------------------- - - 1. When using the partial match feature with pcre_dfa_exec(), it was not - anchoring the second and subsequent partial matches at the new starting - point. This could lead to incorrect results. For example, with the pattern - /1234/, partially matching against "123" and then "a4" gave a match. - - 2. Changes to pcregrep: - - (a) All non-match returns from pcre_exec() were being treated as failures - to match the line. Now, unless the error is PCRE_ERROR_NOMATCH, an - error message is output. Some extra information is given for the - PCRE_ERROR_MATCHLIMIT and PCRE_ERROR_RECURSIONLIMIT errors, which are - probably the only errors that are likely to be caused by users (by - specifying a regex that has nested indefinite repeats, for instance). - If there are more than 20 of these errors, pcregrep is abandoned. - - (b) A binary zero was treated as data while matching, but terminated the - output line if it was written out. This has been fixed: binary zeroes - are now no different to any other data bytes. - - (c) Whichever of the LC_ALL or LC_CTYPE environment variables is set is - used to set a locale for matching. The --locale=xxxx long option has - been added (no short equivalent) to specify a locale explicitly on the - pcregrep command, overriding the environment variables. - - (d) When -B was used with -n, some line numbers in the output were one less - than they should have been. - - (e) Added the -o (--only-matching) option. - - (f) If -A or -C was used with -c (count only), some lines of context were - accidentally printed for the final match. - - (g) Added the -H (--with-filename) option. - - (h) The combination of options -rh failed to suppress file names for files - that were found from directory arguments. - - (i) Added the -D (--devices) and -d (--directories) options. - - (j) Added the -F (--fixed-strings) option. - - (k) Allow "-" to be used as a file name for -f as well as for a data file. - - (l) Added the --colo(u)r option. - - (m) Added Jeffrey Friedl's -S testing option, but within #ifdefs so that it - is not present by default. - - 3. A nasty bug was discovered in the handling of recursive patterns, that is, - items such as (?R) or (?1), when the recursion could match a number of - alternatives. If it matched one of the alternatives, but subsequently, - outside the recursion, there was a failure, the code tried to back up into - the recursion. However, because of the way PCRE is implemented, this is not - possible, and the result was an incorrect result from the match. - - In order to prevent this happening, the specification of recursion has - been changed so that all such subpatterns are automatically treated as - atomic groups. Thus, for example, (?R) is treated as if it were (?>(?R)). - - 4. I had overlooked the fact that, in some locales, there are characters for - which isalpha() is true but neither isupper() nor islower() are true. In - the fr_FR locale, for instance, the \xAA and \xBA characters (ordmasculine - and ordfeminine) are like this. This affected the treatment of \w and \W - when they appeared in character classes, but not when they appeared outside - a character class. The bit map for "word" characters is now created - separately from the results of isalnum() instead of just taking it from the - upper, lower, and digit maps. (Plus the underscore character, of course.) - - 5. The above bug also affected the handling of POSIX character classes such as - [[:alpha:]] and [[:alnum:]]. These do not have their own bit maps in PCRE's - permanent tables. Instead, the bit maps for such a class were previously - created as the appropriate unions of the upper, lower, and digit bitmaps. - Now they are created by subtraction from the [[:word:]] class, which has - its own bitmap. - - 6. The [[:blank:]] character class matches horizontal, but not vertical space. - It is created by subtracting the vertical space characters (\x09, \x0a, - \x0b, \x0c) from the [[:space:]] bitmap. Previously, however, the - subtraction was done in the overall bitmap for a character class, meaning - that a class such as [\x0c[:blank:]] was incorrect because \x0c would not - be recognized. This bug has been fixed. - - 7. Patches from the folks at Google: - - (a) pcrecpp.cc: "to handle a corner case that may or may not happen in - real life, but is still worth protecting against". - - (b) pcrecpp.cc: "corrects a bug when negative radixes are used with - regular expressions". - - (c) pcre_scanner.cc: avoid use of std::count() because not all systems - have it. - - (d) Split off pcrecpparg.h from pcrecpp.h and had the former built by - "configure" and the latter not, in order to fix a problem somebody had - with compiling the Arg class on HP-UX. - - (e) Improve the error-handling of the C++ wrapper a little bit. - - (f) New tests for checking recursion limiting. - - 8. The pcre_memmove() function, which is used only if the environment does not - have a standard memmove() function (and is therefore rarely compiled), - contained two bugs: (a) use of int instead of size_t, and (b) it was not - returning a result (though PCRE never actually uses the result). - - 9. In the POSIX regexec() interface, if nmatch is specified as a ridiculously - large number - greater than INT_MAX/(3*sizeof(int)) - REG_ESPACE is - returned instead of calling malloc() with an overflowing number that would - most likely cause subsequent chaos. - -10. The debugging option of pcretest was not showing the NO_AUTO_CAPTURE flag. - -11. The POSIX flag REG_NOSUB is now supported. When a pattern that was compiled - with this option is matched, the nmatch and pmatch options of regexec() are - ignored. - -12. Added REG_UTF8 to the POSIX interface. This is not defined by POSIX, but is - provided in case anyone wants to the the POSIX interface with UTF-8 - strings. - -13. Added CXXLDFLAGS to the Makefile parameters to provide settings only on the - C++ linking (needed for some HP-UX environments). - -14. Avoid compiler warnings in get_ucpname() when compiled without UCP support - (unused parameter) and in the pcre_printint() function (omitted "default" - switch label when the default is to do nothing). - -15. Added some code to make it possible, when PCRE is compiled as a C++ - library, to replace subject pointers for pcre_exec() with a smart pointer - class, thus making it possible to process discontinuous strings. - -16. The two macros PCRE_EXPORT and PCRE_DATA_SCOPE are confusing, and perform - much the same function. They were added by different people who were trying - to make PCRE easy to compile on non-Unix systems. It has been suggested - that PCRE_EXPORT be abolished now that there is more automatic apparatus - for compiling on Windows systems. I have therefore replaced it with - PCRE_DATA_SCOPE. This is set automatically for Windows; if not set it - defaults to "extern" for C or "extern C" for C++, which works fine on - Unix-like systems. It is now possible to override the value of PCRE_DATA_ - SCOPE with something explicit in config.h. In addition: - - (a) pcreposix.h still had just "extern" instead of either of these macros; - I have replaced it with PCRE_DATA_SCOPE. - - (b) Functions such as _pcre_xclass(), which are internal to the library, - but external in the C sense, all had PCRE_EXPORT in their definitions. - This is apparently wrong for the Windows case, so I have removed it. - (It makes no difference on Unix-like systems.) - -17. Added a new limit, MATCH_LIMIT_RECURSION, which limits the depth of nesting - of recursive calls to match(). This is different to MATCH_LIMIT because - that limits the total number of calls to match(), not all of which increase - the depth of recursion. Limiting the recursion depth limits the amount of - stack (or heap if NO_RECURSE is set) that is used. The default can be set - when PCRE is compiled, and changed at run time. A patch from Google adds - this functionality to the C++ interface. - -18. Changes to the handling of Unicode character properties: - - (a) Updated the table to Unicode 4.1.0. - - (b) Recognize characters that are not in the table as "Cn" (undefined). - - (c) I revised the way the table is implemented to a much improved format - which includes recognition of ranges. It now supports the ranges that - are defined in UnicodeData.txt, and it also amalgamates other - characters into ranges. This has reduced the number of entries in the - table from around 16,000 to around 3,000, thus reducing its size - considerably. I realized I did not need to use a tree structure after - all - a binary chop search is just as efficient. Having reduced the - number of entries, I extended their size from 6 bytes to 8 bytes to - allow for more data. - - (d) Added support for Unicode script names via properties such as \p{Han}. - -19. In UTF-8 mode, a backslash followed by a non-Ascii character was not - matching that character. - -20. When matching a repeated Unicode property with a minimum greater than zero, - (for example \pL{2,}), PCRE could look past the end of the subject if it - reached it while seeking the minimum number of characters. This could - happen only if some of the characters were more than one byte long, because - there is a check for at least the minimum number of bytes. - -21. Refactored the implementation of \p and \P so as to be more general, to - allow for more different types of property in future. This has changed the - compiled form incompatibly. Anybody with saved compiled patterns that use - \p or \P will have to recompile them. - -22. Added "Any" and "L&" to the supported property types. - -23. Recognize \x{...} as a code point specifier, even when not in UTF-8 mode, - but give a compile time error if the value is greater than 0xff. - -24. The man pages for pcrepartial, pcreprecompile, and pcre_compile2 were - accidentally not being installed or uninstalled. - -25. The pcre.h file was built from pcre.h.in, but the only changes that were - made were to insert the current release number. This seemed silly, because - it made things harder for people building PCRE on systems that don't run - "configure". I have turned pcre.h into a distributed file, no longer built - by "configure", with the version identification directly included. There is - no longer a pcre.h.in file. - - However, this change necessitated a change to the pcre-config script as - well. It is built from pcre-config.in, and one of the substitutions was the - release number. I have updated configure.ac so that ./configure now finds - the release number by grepping pcre.h. - -26. Added the ability to run the tests under valgrind. - - -Version 6.4 05-Sep-05 ---------------------- - - 1. Change 6.0/10/(l) to pcregrep introduced a bug that caused separator lines - "--" to be printed when multiple files were scanned, even when none of the - -A, -B, or -C options were used. This is not compatible with Gnu grep, so I - consider it to be a bug, and have restored the previous behaviour. - - 2. A couple of code tidies to get rid of compiler warnings. - - 3. The pcretest program used to cheat by referring to symbols in the library - whose names begin with _pcre_. These are internal symbols that are not - really supposed to be visible externally, and in some environments it is - possible to suppress them. The cheating is now confined to including - certain files from the library's source, which is a bit cleaner. - - 4. Renamed pcre.in as pcre.h.in to go with pcrecpp.h.in; it also makes the - file's purpose clearer. - - 5. Reorganized pcre_ucp_findchar(). - - -Version 6.3 15-Aug-05 ---------------------- - - 1. The file libpcre.pc.in did not have general read permission in the tarball. - - 2. There were some problems when building without C++ support: - - (a) If C++ support was not built, "make install" and "make test" still - tried to test it. - - (b) There were problems when the value of CXX was explicitly set. Some - changes have been made to try to fix these, and ... - - (c) --disable-cpp can now be used to explicitly disable C++ support. - - (d) The use of @CPP_OBJ@ directly caused a blank line preceded by a - backslash in a target when C++ was disabled. This confuses some - versions of "make", apparently. Using an intermediate variable solves - this. (Same for CPP_LOBJ.) - - 3. $(LINK_FOR_BUILD) now includes $(CFLAGS_FOR_BUILD) and $(LINK) - (non-Windows) now includes $(CFLAGS) because these flags are sometimes - necessary on certain architectures. - - 4. Added a setting of -export-symbols-regex to the link command to remove - those symbols that are exported in the C sense, but actually are local - within the library, and not documented. Their names all begin with - "_pcre_". This is not a perfect job, because (a) we have to except some - symbols that pcretest ("illegally") uses, and (b) the facility isn't always - available (and never for static libraries). I have made a note to try to - find a way round (a) in the future. - - -Version 6.2 01-Aug-05 ---------------------- - - 1. There was no test for integer overflow of quantifier values. A construction - such as {1111111111111111} would give undefined results. What is worse, if - a minimum quantifier for a parenthesized subpattern overflowed and became - negative, the calculation of the memory size went wrong. This could have - led to memory overwriting. - - 2. Building PCRE using VPATH was broken. Hopefully it is now fixed. - - 3. Added "b" to the 2nd argument of fopen() in dftables.c, for non-Unix-like - operating environments where this matters. - - 4. Applied Giuseppe Maxia's patch to add additional features for controlling - PCRE options from within the C++ wrapper. - - 5. Named capturing subpatterns were not being correctly counted when a pattern - was compiled. This caused two problems: (a) If there were more than 100 - such subpatterns, the calculation of the memory needed for the whole - compiled pattern went wrong, leading to an overflow error. (b) Numerical - back references of the form \12, where the number was greater than 9, were - not recognized as back references, even though there were sufficient - previous subpatterns. - - 6. Two minor patches to pcrecpp.cc in order to allow it to compile on older - versions of gcc, e.g. 2.95.4. - - -Version 6.1 21-Jun-05 ---------------------- - - 1. There was one reference to the variable "posix" in pcretest.c that was not - surrounded by "#if !defined NOPOSIX". - - 2. Make it possible to compile pcretest without DFA support, UTF8 support, or - the cross-check on the old pcre_info() function, for the benefit of the - cut-down version of PCRE that is currently imported into Exim. - - 3. A (silly) pattern starting with (?i)(?-i) caused an internal space - allocation error. I've done the easy fix, which wastes 2 bytes for sensible - patterns that start (?i) but I don't think that matters. The use of (?i) is - just an example; this all applies to the other options as well. - - 4. Since libtool seems to echo the compile commands it is issuing, the output - from "make" can be reduced a bit by putting "@" in front of each libtool - compile command. - - 5. Patch from the folks at Google for configure.in to be a bit more thorough - in checking for a suitable C++ installation before trying to compile the - C++ stuff. This should fix a reported problem when a compiler was present, - but no suitable headers. - - 6. The man pages all had just "PCRE" as their title. I have changed them to - be the relevant file name. I have also arranged that these names are - retained in the file doc/pcre.txt, which is a concatenation in text format - of all the man pages except the little individual ones for each function. - - 7. The NON-UNIX-USE file had not been updated for the different set of source - files that come with release 6. I also added a few comments about the C++ - wrapper. - - -Version 6.0 07-Jun-05 ---------------------- - - 1. Some minor internal re-organization to help with my DFA experiments. - - 2. Some missing #ifdef SUPPORT_UCP conditionals in pcretest and printint that - didn't matter for the library itself when fully configured, but did matter - when compiling without UCP support, or within Exim, where the ucp files are - not imported. - - 3. Refactoring of the library code to split up the various functions into - different source modules. The addition of the new DFA matching code (see - below) to a single monolithic source would have made it really too - unwieldy, quite apart from causing all the code to be include in a - statically linked application, when only some functions are used. This is - relevant even without the DFA addition now that patterns can be compiled in - one application and matched in another. - - The downside of splitting up is that there have to be some external - functions and data tables that are used internally in different modules of - the library but which are not part of the API. These have all had their - names changed to start with "_pcre_" so that they are unlikely to clash - with other external names. - - 4. Added an alternate matching function, pcre_dfa_exec(), which matches using - a different (DFA) algorithm. Although it is slower than the original - function, it does have some advantages for certain types of matching - problem. - - 5. Upgrades to pcretest in order to test the features of pcre_dfa_exec(), - including restarting after a partial match. - - 6. A patch for pcregrep that defines INVALID_FILE_ATTRIBUTES if it is not - defined when compiling for Windows was sent to me. I have put it into the - code, though I have no means of testing or verifying it. - - 7. Added the pcre_refcount() auxiliary function. - - 8. Added the PCRE_FIRSTLINE option. This constrains an unanchored pattern to - match before or at the first newline in the subject string. In pcretest, - the /f option on a pattern can be used to set this. - - 9. A repeated \w when used in UTF-8 mode with characters greater than 256 - would behave wrongly. This has been present in PCRE since release 4.0. - -10. A number of changes to the pcregrep command: - - (a) Refactored how -x works; insert ^(...)$ instead of setting - PCRE_ANCHORED and checking the length, in preparation for adding - something similar for -w. - - (b) Added the -w (match as a word) option. - - (c) Refactored the way lines are read and buffered so as to have more - than one at a time available. - - (d) Implemented a pcregrep test script. - - (e) Added the -M (multiline match) option. This allows patterns to match - over several lines of the subject. The buffering ensures that at least - 8K, or the rest of the document (whichever is the shorter) is available - for matching (and similarly the previous 8K for lookbehind assertions). - - (f) Changed the --help output so that it now says - - -w, --word-regex(p) - - instead of two lines, one with "regex" and the other with "regexp" - because that confused at least one person since the short forms are the - same. (This required a bit of code, as the output is generated - automatically from a table. It wasn't just a text change.) - - (g) -- can be used to terminate pcregrep options if the next thing isn't an - option but starts with a hyphen. Could be a pattern or a path name - starting with a hyphen, for instance. - - (h) "-" can be given as a file name to represent stdin. - - (i) When file names are being printed, "(standard input)" is used for - the standard input, for compatibility with GNU grep. Previously - "<stdin>" was used. - - (j) The option --label=xxx can be used to supply a name to be used for - stdin when file names are being printed. There is no short form. - - (k) Re-factored the options decoding logic because we are going to add - two more options that take data. Such options can now be given in four - different ways, e.g. "-fname", "-f name", "--file=name", "--file name". - - (l) Added the -A, -B, and -C options for requesting that lines of context - around matches be printed. - - (m) Added the -L option to print the names of files that do not contain - any matching lines, that is, the complement of -l. - - (n) The return code is 2 if any file cannot be opened, but pcregrep does - continue to scan other files. - - (o) The -s option was incorrectly implemented. For compatibility with other - greps, it now suppresses the error message for a non-existent or non- - accessible file (but not the return code). There is a new option called - -q that suppresses the output of matching lines, which was what -s was - previously doing. - - (p) Added --include and --exclude options to specify files for inclusion - and exclusion when recursing. - -11. The Makefile was not using the Autoconf-supported LDFLAGS macro properly. - Hopefully, it now does. - -12. Missing cast in pcre_study(). - -13. Added an "uninstall" target to the makefile. - -14. Replaced "extern" in the function prototypes in Makefile.in with - "PCRE_DATA_SCOPE", which defaults to 'extern' or 'extern "C"' in the Unix - world, but is set differently for Windows. - -15. Added a second compiling function called pcre_compile2(). The only - difference is that it has an extra argument, which is a pointer to an - integer error code. When there is a compile-time failure, this is set - non-zero, in addition to the error test pointer being set to point to an - error message. The new argument may be NULL if no error number is required - (but then you may as well call pcre_compile(), which is now just a - wrapper). This facility is provided because some applications need a - numeric error indication, but it has also enabled me to tidy up the way - compile-time errors are handled in the POSIX wrapper. - -16. Added VPATH=.libs to the makefile; this should help when building with one - prefix path and installing with another. (Or so I'm told by someone who - knows more about this stuff than I do.) - -17. Added a new option, REG_DOTALL, to the POSIX function regcomp(). This - passes PCRE_DOTALL to the pcre_compile() function, making the "." character - match everything, including newlines. This is not POSIX-compatible, but - somebody wanted the feature. From pcretest it can be activated by using - both the P and the s flags. - -18. AC_PROG_LIBTOOL appeared twice in Makefile.in. Removed one. - -19. libpcre.pc was being incorrectly installed as executable. - -20. A couple of places in pcretest check for end-of-line by looking for '\n'; - it now also looks for '\r' so that it will work unmodified on Windows. - -21. Added Google's contributed C++ wrapper to the distribution. - -22. Added some untidy missing memory free() calls in pcretest, to keep - Electric Fence happy when testing. - - - -Version 5.0 13-Sep-04 ---------------------- - - 1. Internal change: literal characters are no longer packed up into items - containing multiple characters in a single byte-string. Each character - is now matched using a separate opcode. However, there may be more than one - byte in the character in UTF-8 mode. - - 2. The pcre_callout_block structure has two new fields: pattern_position and - next_item_length. These contain the offset in the pattern to the next match - item, and its length, respectively. - - 3. The PCRE_AUTO_CALLOUT option for pcre_compile() requests the automatic - insertion of callouts before each pattern item. Added the /C option to - pcretest to make use of this. - - 4. On the advice of a Windows user, the lines - - #if defined(_WIN32) || defined(WIN32) - _setmode( _fileno( stdout ), 0x8000 ); - #endif /* defined(_WIN32) || defined(WIN32) */ - - have been added to the source of pcretest. This apparently does useful - magic in relation to line terminators. - - 5. Changed "r" and "w" in the calls to fopen() in pcretest to "rb" and "wb" - for the benefit of those environments where the "b" makes a difference. - - 6. The icc compiler has the same options as gcc, but "configure" doesn't seem - to know about it. I have put a hack into configure.in that adds in code - to set GCC=yes if CC=icc. This seems to end up at a point in the - generated configure script that is early enough to affect the setting of - compiler options, which is what is needed, but I have no means of testing - whether it really works. (The user who reported this had patched the - generated configure script, which of course I cannot do.) - - LATER: After change 22 below (new libtool files), the configure script - seems to know about icc (and also ecc). Therefore, I have commented out - this hack in configure.in. - - 7. Added support for pkg-config (2 patches were sent in). - - 8. Negated POSIX character classes that used a combination of internal tables - were completely broken. These were [[:^alpha:]], [[:^alnum:]], and - [[:^ascii]]. Typically, they would match almost any characters. The other - POSIX classes were not broken in this way. - - 9. Matching the pattern "\b.*?" against "ab cd", starting at offset 1, failed - to find the match, as PCRE was deluded into thinking that the match had to - start at the start point or following a newline. The same bug applied to - patterns with negative forward assertions or any backward assertions - preceding ".*" at the start, unless the pattern required a fixed first - character. This was a failing pattern: "(?!.bcd).*". The bug is now fixed. - -10. In UTF-8 mode, when moving forwards in the subject after a failed match - starting at the last subject character, bytes beyond the end of the subject - string were read. - -11. Renamed the variable "class" as "classbits" to make life easier for C++ - users. (Previously there was a macro definition, but it apparently wasn't - enough.) - -12. Added the new field "tables" to the extra data so that tables can be passed - in at exec time, or the internal tables can be re-selected. This allows - a compiled regex to be saved and re-used at a later time by a different - program that might have everything at different addresses. - -13. Modified the pcre-config script so that, when run on Solaris, it shows a - -R library as well as a -L library. - -14. The debugging options of pcretest (-d on the command line or D on a - pattern) showed incorrect output for anything following an extended class - that contained multibyte characters and which was followed by a quantifier. - -15. Added optional support for general category Unicode character properties - via the \p, \P, and \X escapes. Unicode property support implies UTF-8 - support. It adds about 90K to the size of the library. The meanings of the - inbuilt class escapes such as \d and \s have NOT been changed. - -16. Updated pcredemo.c to include calls to free() to release the memory for the - compiled pattern. - -17. The generated file chartables.c was being created in the source directory - instead of in the building directory. This caused the build to fail if the - source directory was different from the building directory, and was - read-only. - -18. Added some sample Win commands from Mark Tetrode into the NON-UNIX-USE - file. No doubt somebody will tell me if they don't make sense... Also added - Dan Mooney's comments about building on OpenVMS. - -19. Added support for partial matching via the PCRE_PARTIAL option for - pcre_exec() and the \P data escape in pcretest. - -20. Extended pcretest with 3 new pattern features: - - (i) A pattern option of the form ">rest-of-line" causes pcretest to - write the compiled pattern to the file whose name is "rest-of-line". - This is a straight binary dump of the data, with the saved pointer to - the character tables forced to be NULL. The study data, if any, is - written too. After writing, pcretest reads a new pattern. - - (ii) If, instead of a pattern, "<rest-of-line" is given, pcretest reads a - compiled pattern from the given file. There must not be any - occurrences of "<" in the file name (pretty unlikely); if there are, - pcretest will instead treat the initial "<" as a pattern delimiter. - After reading in the pattern, pcretest goes on to read data lines as - usual. - - (iii) The F pattern option causes pcretest to flip the bytes in the 32-bit - and 16-bit fields in a compiled pattern, to simulate a pattern that - was compiled on a host of opposite endianness. - -21. The pcre-exec() function can now cope with patterns that were compiled on - hosts of opposite endianness, with this restriction: - - As for any compiled expression that is saved and used later, the tables - pointer field cannot be preserved; the extra_data field in the arguments - to pcre_exec() should be used to pass in a tables address if a value - other than the default internal tables were used at compile time. - -22. Calling pcre_exec() with a negative value of the "ovecsize" parameter is - now diagnosed as an error. Previously, most of the time, a negative number - would have been treated as zero, but if in addition "ovector" was passed as - NULL, a crash could occur. - -23. Updated the files ltmain.sh, config.sub, config.guess, and aclocal.m4 with - new versions from the libtool 1.5 distribution (the last one is a copy of - a file called libtool.m4). This seems to have fixed the need to patch - "configure" to support Darwin 1.3 (which I used to do). However, I still - had to patch ltmain.sh to ensure that ${SED} is set (it isn't on my - workstation). - -24. Changed the PCRE licence to be the more standard "BSD" licence. - - -Version 4.5 01-Dec-03 ---------------------- - - 1. There has been some re-arrangement of the code for the match() function so - that it can be compiled in a version that does not call itself recursively. - Instead, it keeps those local variables that need separate instances for - each "recursion" in a frame on the heap, and gets/frees frames whenever it - needs to "recurse". Keeping track of where control must go is done by means - of setjmp/longjmp. The whole thing is implemented by a set of macros that - hide most of the details from the main code, and operates only if - NO_RECURSE is defined while compiling pcre.c. If PCRE is built using the - "configure" mechanism, "--disable-stack-for-recursion" turns on this way of - operating. - - To make it easier for callers to provide specially tailored get/free - functions for this usage, two new functions, pcre_stack_malloc, and - pcre_stack_free, are used. They are always called in strict stacking order, - and the size of block requested is always the same. - - The PCRE_CONFIG_STACKRECURSE info parameter can be used to find out whether - PCRE has been compiled to use the stack or the heap for recursion. The - -C option of pcretest uses this to show which version is compiled. - - A new data escape \S, is added to pcretest; it causes the amounts of store - obtained and freed by both kinds of malloc/free at match time to be added - to the output. - - 2. Changed the locale test to use "fr_FR" instead of "fr" because that's - what's available on my current Linux desktop machine. - - 3. When matching a UTF-8 string, the test for a valid string at the start has - been extended. If start_offset is not zero, PCRE now checks that it points - to a byte that is the start of a UTF-8 character. If not, it returns - PCRE_ERROR_BADUTF8_OFFSET (-11). Note: the whole string is still checked; - this is necessary because there may be backward assertions in the pattern. - When matching the same subject several times, it may save resources to use - PCRE_NO_UTF8_CHECK on all but the first call if the string is long. - - 4. The code for checking the validity of UTF-8 strings has been tightened so - that it rejects (a) strings containing 0xfe or 0xff bytes and (b) strings - containing "overlong sequences". - - 5. Fixed a bug (appearing twice) that I could not find any way of exploiting! - I had written "if ((digitab[*p++] && chtab_digit) == 0)" where the "&&" - should have been "&", but it just so happened that all the cases this let - through by mistake were picked up later in the function. - - 6. I had used a variable called "isblank" - this is a C99 function, causing - some compilers to warn. To avoid this, I renamed it (as "blankclass"). - - 7. Cosmetic: (a) only output another newline at the end of pcretest if it is - prompting; (b) run "./pcretest /dev/null" at the start of the test script - so the version is shown; (c) stop "make test" echoing "./RunTest". - - 8. Added patches from David Burgess to enable PCRE to run on EBCDIC systems. - - 9. The prototype for memmove() for systems that don't have it was using - size_t, but the inclusion of the header that defines size_t was later. I've - moved the #includes for the C headers earlier to avoid this. - -10. Added some adjustments to the code to make it easier to compiler on certain - special systems: - - (a) Some "const" qualifiers were missing. - (b) Added the macro EXPORT before all exported functions; by default this - is defined to be empty. - (c) Changed the dftables auxiliary program (that builds chartables.c) so - that it reads its output file name as an argument instead of writing - to the standard output and assuming this can be redirected. - -11. In UTF-8 mode, if a recursive reference (e.g. (?1)) followed a character - class containing characters with values greater than 255, PCRE compilation - went into a loop. - -12. A recursive reference to a subpattern that was within another subpattern - that had a minimum quantifier of zero caused PCRE to crash. For example, - (x(y(?2))z)? provoked this bug with a subject that got as far as the - recursion. If the recursively-called subpattern itself had a zero repeat, - that was OK. - -13. In pcretest, the buffer for reading a data line was set at 30K, but the - buffer into which it was copied (for escape processing) was still set at - 1024, so long lines caused crashes. - -14. A pattern such as /[ab]{1,3}+/ failed to compile, giving the error - "internal error: code overflow...". This applied to any character class - that was followed by a possessive quantifier. - -15. Modified the Makefile to add libpcre.la as a prerequisite for - libpcreposix.la because I was told this is needed for a parallel build to - work. - -16. If a pattern that contained .* following optional items at the start was - studied, the wrong optimizing data was generated, leading to matching - errors. For example, studying /[ab]*.*c/ concluded, erroneously, that any - matching string must start with a or b or c. The correct conclusion for - this pattern is that a match can start with any character. - - -Version 4.4 13-Aug-03 ---------------------- - - 1. In UTF-8 mode, a character class containing characters with values between - 127 and 255 was not handled correctly if the compiled pattern was studied. - In fixing this, I have also improved the studying algorithm for such - classes (slightly). - - 2. Three internal functions had redundant arguments passed to them. Removal - might give a very teeny performance improvement. - - 3. Documentation bug: the value of the capture_top field in a callout is *one - more than* the number of the hightest numbered captured substring. - - 4. The Makefile linked pcretest and pcregrep with -lpcre, which could result - in incorrectly linking with a previously installed version. They now link - explicitly with libpcre.la. - - 5. configure.in no longer needs to recognize Cygwin specially. - - 6. A problem in pcre.in for Windows platforms is fixed. - - 7. If a pattern was successfully studied, and the -d (or /D) flag was given to - pcretest, it used to include the size of the study block as part of its - output. Unfortunately, the structure contains a field that has a different - size on different hardware architectures. This meant that the tests that - showed this size failed. As the block is currently always of a fixed size, - this information isn't actually particularly useful in pcretest output, so - I have just removed it. - - 8. Three pre-processor statements accidentally did not start in column 1. - Sadly, there are *still* compilers around that complain, even though - standard C has not required this for well over a decade. Sigh. - - 9. In pcretest, the code for checking callouts passed small integers in the - callout_data field, which is a void * field. However, some picky compilers - complained about the casts involved for this on 64-bit systems. Now - pcretest passes the address of the small integer instead, which should get - rid of the warnings. - -10. By default, when in UTF-8 mode, PCRE now checks for valid UTF-8 strings at - both compile and run time, and gives an error if an invalid UTF-8 sequence - is found. There is a option for disabling this check in cases where the - string is known to be correct and/or the maximum performance is wanted. - -11. In response to a bug report, I changed one line in Makefile.in from - - -Wl,--out-implib,.libs/lib@WIN_PREFIX@pcreposix.dll.a \ - to - -Wl,--out-implib,.libs/@WIN_PREFIX@libpcreposix.dll.a \ - - to look similar to other lines, but I have no way of telling whether this - is the right thing to do, as I do not use Windows. No doubt I'll get told - if it's wrong... - - -Version 4.3 21-May-03 ---------------------- - -1. Two instances of @WIN_PREFIX@ omitted from the Windows targets in the - Makefile. - -2. Some refactoring to improve the quality of the code: - - (i) The utf8_table... variables are now declared "const". - - (ii) The code for \cx, which used the "case flipping" table to upper case - lower case letters, now just substracts 32. This is ASCII-specific, - but the whole concept of \cx is ASCII-specific, so it seems - reasonable. - - (iii) PCRE was using its character types table to recognize decimal and - hexadecimal digits in the pattern. This is silly, because it handles - only 0-9, a-f, and A-F, but the character types table is locale- - specific, which means strange things might happen. A private - table is now used for this - though it costs 256 bytes, a table is - much faster than multiple explicit tests. Of course, the standard - character types table is still used for matching digits in subject - strings against \d. - - (iv) Strictly, the identifier ESC_t is reserved by POSIX (all identifiers - ending in _t are). So I've renamed it as ESC_tee. - -3. The first argument for regexec() in the POSIX wrapper should have been - defined as "const". - -4. Changed pcretest to use malloc() for its buffers so that they can be - Electric Fenced for debugging. - -5. There were several places in the code where, in UTF-8 mode, PCRE would try - to read one or more bytes before the start of the subject string. Often this - had no effect on PCRE's behaviour, but in some circumstances it could - provoke a segmentation fault. - -6. A lookbehind at the start of a pattern in UTF-8 mode could also cause PCRE - to try to read one or more bytes before the start of the subject string. - -7. A lookbehind in a pattern matched in non-UTF-8 mode on a PCRE compiled with - UTF-8 support could misbehave in various ways if the subject string - contained bytes with the 0x80 bit set and the 0x40 bit unset in a lookbehind - area. (PCRE was not checking for the UTF-8 mode flag, and trying to move - back over UTF-8 characters.) - - -Version 4.2 14-Apr-03 ---------------------- - -1. Typo "#if SUPPORT_UTF8" instead of "#ifdef SUPPORT_UTF8" fixed. - -2. Changes to the building process, supplied by Ronald Landheer-Cieslak - [ON_WINDOWS]: new variable, "#" on non-Windows platforms - [NOT_ON_WINDOWS]: new variable, "#" on Windows platforms - [WIN_PREFIX]: new variable, "cyg" for Cygwin - * Makefile.in: use autoconf substitution for OBJEXT, EXEEXT, BUILD_OBJEXT - and BUILD_EXEEXT - Note: automatic setting of the BUILD variables is not yet working - set CPPFLAGS and BUILD_CPPFLAGS (but don't use yet) - should be used at - compile-time but not at link-time - [LINK]: use for linking executables only - make different versions for Windows and non-Windows - [LINKLIB]: new variable, copy of UNIX-style LINK, used for linking - libraries - [LINK_FOR_BUILD]: new variable - [OBJEXT]: use throughout - [EXEEXT]: use throughout - <winshared>: new target - <wininstall>: new target - <dftables.o>: use native compiler - <dftables>: use native linker - <install>: handle Windows platform correctly - <clean>: ditto - <check>: ditto - copy DLL to top builddir before testing - - As part of these changes, -no-undefined was removed again. This was reported - to give trouble on HP-UX 11.0, so getting rid of it seems like a good idea - in any case. - -3. Some tidies to get rid of compiler warnings: - - . In the match_data structure, match_limit was an unsigned long int, whereas - match_call_count was an int. I've made them both unsigned long ints. - - . In pcretest the fact that a const uschar * doesn't automatically cast to - a void * provoked a warning. - - . Turning on some more compiler warnings threw up some "shadow" variables - and a few more missing casts. - -4. If PCRE was complied with UTF-8 support, but called without the PCRE_UTF8 - option, a class that contained a single character with a value between 128 - and 255 (e.g. /[\xFF]/) caused PCRE to crash. - -5. If PCRE was compiled with UTF-8 support, but called without the PCRE_UTF8 - option, a class that contained several characters, but with at least one - whose value was between 128 and 255 caused PCRE to crash. - - -Version 4.1 12-Mar-03 ---------------------- - -1. Compiling with gcc -pedantic found a couple of places where casts were -needed, and a string in dftables.c that was longer than standard compilers are -required to support. - -2. Compiling with Sun's compiler found a few more places where the code could -be tidied up in order to avoid warnings. - -3. The variables for cross-compiling were called HOST_CC and HOST_CFLAGS; the -first of these names is deprecated in the latest Autoconf in favour of the name -CC_FOR_BUILD, because "host" is typically used to mean the system on which the -compiled code will be run. I can't find a reference for HOST_CFLAGS, but by -analogy I have changed it to CFLAGS_FOR_BUILD. - -4. Added -no-undefined to the linking command in the Makefile, because this is -apparently helpful for Windows. To make it work, also added "-L. -lpcre" to the -linking step for the pcreposix library. - -5. PCRE was failing to diagnose the case of two named groups with the same -name. - -6. A problem with one of PCRE's optimizations was discovered. PCRE remembers a -literal character that is needed in the subject for a match, and scans along to -ensure that it is present before embarking on the full matching process. This -saves time in cases of nested unlimited repeats that are never going to match. -Problem: the scan can take a lot of time if the subject is very long (e.g. -megabytes), thus penalizing straightforward matches. It is now done only if the -amount of subject to be scanned is less than 1000 bytes. - -7. A lesser problem with the same optimization is that it was recording the -first character of an anchored pattern as "needed", thus provoking a search -right along the subject, even when the first match of the pattern was going to -fail. The "needed" character is now not set for anchored patterns, unless it -follows something in the pattern that is of non-fixed length. Thus, it still -fulfils its original purpose of finding quick non-matches in cases of nested -unlimited repeats, but isn't used for simple anchored patterns such as /^abc/. - - -Version 4.0 17-Feb-03 ---------------------- - -1. If a comment in an extended regex that started immediately after a meta-item -extended to the end of string, PCRE compiled incorrect data. This could lead to -all kinds of weird effects. Example: /#/ was bad; /()#/ was bad; /a#/ was not. - -2. Moved to autoconf 2.53 and libtool 1.4.2. - -3. Perl 5.8 no longer needs "use utf8" for doing UTF-8 things. Consequently, -the special perltest8 script is no longer needed - all the tests can be run -from a single perltest script. - -4. From 5.004, Perl has not included the VT character (0x0b) in the set defined -by \s. It has now been removed in PCRE. This means it isn't recognized as -whitespace in /x regexes too, which is the same as Perl. Note that the POSIX -class [:space:] *does* include VT, thereby creating a mess. - -5. Added the class [:blank:] (a GNU extension from Perl 5.8) to match only -space and tab. - -6. Perl 5.005 was a long time ago. It's time to amalgamate the tests that use -its new features into the main test script, reducing the number of scripts. - -7. Perl 5.8 has changed the meaning of patterns like /a(?i)b/. Earlier versions -were backward compatible, and made the (?i) apply to the whole pattern, as if -/i were given. Now it behaves more logically, and applies the option setting -only to what follows. PCRE has been changed to follow suit. However, if it -finds options settings right at the start of the pattern, it extracts them into -the global options, as before. Thus, they show up in the info data. - -8. Added support for the \Q...\E escape sequence. Characters in between are -treated as literals. This is slightly different from Perl in that $ and @ are -also handled as literals inside the quotes. In Perl, they will cause variable -interpolation. Note the following examples: - - Pattern PCRE matches Perl matches - - \Qabc$xyz\E abc$xyz abc followed by the contents of $xyz - \Qabc\$xyz\E abc\$xyz abc\$xyz - \Qabc\E\$\Qxyz\E abc$xyz abc$xyz - -For compatibility with Perl, \Q...\E sequences are recognized inside character -classes as well as outside them. - -9. Re-organized 3 code statements in pcretest to avoid "overflow in -floating-point constant arithmetic" warnings from a Microsoft compiler. Added a -(size_t) cast to one statement in pcretest and one in pcreposix to avoid -signed/unsigned warnings. - -10. SunOS4 doesn't have strtoul(). This was used only for unpicking the -o -option for pcretest, so I've replaced it by a simple function that does just -that job. - -11. pcregrep was ending with code 0 instead of 2 for the commands "pcregrep" or -"pcregrep -". - -12. Added "possessive quantifiers" ?+, *+, ++, and {,}+ which come from Sun's -Java package. This provides some syntactic sugar for simple cases of what my -documentation calls "once-only subpatterns". A pattern such as x*+ is the same -as (?>x*). In other words, if what is inside (?>...) is just a single repeated -item, you can use this simplified notation. Note that only makes sense with -greedy quantifiers. Consequently, the use of the possessive quantifier forces -greediness, whatever the setting of the PCRE_UNGREEDY option. - -13. A change of greediness default within a pattern was not taking effect at -the current level for patterns like /(b+(?U)a+)/. It did apply to parenthesized -subpatterns that followed. Patterns like /b+(?U)a+/ worked because the option -was abstracted outside. - -14. PCRE now supports the \G assertion. It is true when the current matching -position is at the start point of the match. This differs from \A when the -starting offset is non-zero. Used with the /g option of pcretest (or similar -code), it works in the same way as it does for Perl's /g option. If all -alternatives of a regex begin with \G, the expression is anchored to the start -match position, and the "anchored" flag is set in the compiled expression. - -15. Some bugs concerning the handling of certain option changes within patterns -have been fixed. These applied to options other than (?ims). For example, -"a(?x: b c )d" did not match "XabcdY" but did match "Xa b c dY". It should have -been the other way round. Some of this was related to change 7 above. - -16. PCRE now gives errors for /[.x.]/ and /[=x=]/ as unsupported POSIX -features, as Perl does. Previously, PCRE gave the warnings only for /[[.x.]]/ -and /[[=x=]]/. PCRE now also gives an error for /[:name:]/ because it supports -POSIX classes only within a class (e.g. /[[:alpha:]]/). - -17. Added support for Perl's \C escape. This matches one byte, even in UTF8 -mode. Unlike ".", it always matches newline, whatever the setting of -PCRE_DOTALL. However, PCRE does not permit \C to appear in lookbehind -assertions. Perl allows it, but it doesn't (in general) work because it can't -calculate the length of the lookbehind. At least, that's the case for Perl -5.8.0 - I've been told they are going to document that it doesn't work in -future. - -18. Added an error diagnosis for escapes that PCRE does not support: these are -\L, \l, \N, \P, \p, \U, \u, and \X. - -19. Although correctly diagnosing a missing ']' in a character class, PCRE was -reading past the end of the pattern in cases such as /[abcd/. - -20. PCRE was getting more memory than necessary for patterns with classes that -contained both POSIX named classes and other characters, e.g. /[[:space:]abc/. - -21. Added some code, conditional on #ifdef VPCOMPAT, to make life easier for -compiling PCRE for use with Virtual Pascal. - -22. Small fix to the Makefile to make it work properly if the build is done -outside the source tree. - -23. Added a new extension: a condition to go with recursion. If a conditional -subpattern starts with (?(R) the "true" branch is used if recursion has -happened, whereas the "false" branch is used only at the top level. - -24. When there was a very long string of literal characters (over 255 bytes -without UTF support, over 250 bytes with UTF support), the computation of how -much memory was required could be incorrect, leading to segfaults or other -strange effects. - -25. PCRE was incorrectly assuming anchoring (either to start of subject or to -start of line for a non-DOTALL pattern) when a pattern started with (.*) and -there was a subsequent back reference to those brackets. This meant that, for -example, /(.*)\d+\1/ failed to match "abc123bc". Unfortunately, it isn't -possible to check for precisely this case. All we can do is abandon the -optimization if .* occurs inside capturing brackets when there are any back -references whatsoever. (See below for a better fix that came later.) - -26. The handling of the optimization for finding the first character of a -non-anchored pattern, and for finding a character that is required later in the -match were failing in some cases. This didn't break the matching; it just -failed to optimize when it could. The way this is done has been re-implemented. - -27. Fixed typo in error message for invalid (?R item (it said "(?p"). - -28. Added a new feature that provides some of the functionality that Perl -provides with (?{...}). The facility is termed a "callout". The way it is done -in PCRE is for the caller to provide an optional function, by setting -pcre_callout to its entry point. Like pcre_malloc and pcre_free, this is a -global variable. By default it is unset, which disables all calling out. To get -the function called, the regex must include (?C) at appropriate points. This -is, in fact, equivalent to (?C0), and any number <= 255 may be given with (?C). -This provides a means of identifying different callout points. When PCRE -reaches such a point in the regex, if pcre_callout has been set, the external -function is called. It is provided with data in a structure called -pcre_callout_block, which is defined in pcre.h. If the function returns 0, -matching continues; if it returns a non-zero value, the match at the current -point fails. However, backtracking will occur if possible. [This was changed -later and other features added - see item 49 below.] - -29. pcretest is upgraded to test the callout functionality. It provides a -callout function that displays information. By default, it shows the start of -the match and the current position in the text. There are some new data escapes -to vary what happens: - - \C+ in addition, show current contents of captured substrings - \C- do not supply a callout function - \C!n return 1 when callout number n is reached - \C!n!m return 1 when callout number n is reached for the mth time - -30. If pcregrep was called with the -l option and just a single file name, it -output "<stdin>" if a match was found, instead of the file name. - -31. Improve the efficiency of the POSIX API to PCRE. If the number of capturing -slots is less than POSIX_MALLOC_THRESHOLD, use a block on the stack to pass to -pcre_exec(). This saves a malloc/free per call. The default value of -POSIX_MALLOC_THRESHOLD is 10; it can be changed by --with-posix-malloc-threshold -when configuring. - -32. The default maximum size of a compiled pattern is 64K. There have been a -few cases of people hitting this limit. The code now uses macros to handle the -storing of links as offsets within the compiled pattern. It defaults to 2-byte -links, but this can be changed to 3 or 4 bytes by --with-link-size when -configuring. Tests 2 and 5 work only with 2-byte links because they output -debugging information about compiled patterns. - -33. Internal code re-arrangements: - -(a) Moved the debugging function for printing out a compiled regex into - its own source file (printint.c) and used #include to pull it into - pcretest.c and, when DEBUG is defined, into pcre.c, instead of having two - separate copies. - -(b) Defined the list of op-code names for debugging as a macro in - internal.h so that it is next to the definition of the opcodes. - -(c) Defined a table of op-code lengths for simpler skipping along compiled - code. This is again a macro in internal.h so that it is next to the - definition of the opcodes. - -34. Added support for recursive calls to individual subpatterns, along the -lines of Robin Houston's patch (but implemented somewhat differently). - -35. Further mods to the Makefile to help Win32. Also, added code to pcregrep to -allow it to read and process whole directories in Win32. This code was -contributed by Lionel Fourquaux; it has not been tested by me. - -36. Added support for named subpatterns. The Python syntax (?P<name>...) is -used to name a group. Names consist of alphanumerics and underscores, and must -be unique. Back references use the syntax (?P=name) and recursive calls use -(?P>name) which is a PCRE extension to the Python extension. Groups still have -numbers. The function pcre_fullinfo() can be used after compilation to extract -a name/number map. There are three relevant calls: - - PCRE_INFO_NAMEENTRYSIZE yields the size of each entry in the map - PCRE_INFO_NAMECOUNT yields the number of entries - PCRE_INFO_NAMETABLE yields a pointer to the map. - -The map is a vector of fixed-size entries. The size of each entry depends on -the length of the longest name used. The first two bytes of each entry are the -group number, most significant byte first. There follows the corresponding -name, zero terminated. The names are in alphabetical order. - -37. Make the maximum literal string in the compiled code 250 for the non-UTF-8 -case instead of 255. Making it the same both with and without UTF-8 support -means that the same test output works with both. - -38. There was a case of malloc(0) in the POSIX testing code in pcretest. Avoid -calling malloc() with a zero argument. - -39. Change 25 above had to resort to a heavy-handed test for the .* anchoring -optimization. I've improved things by keeping a bitmap of backreferences with -numbers 1-31 so that if .* occurs inside capturing brackets that are not in -fact referenced, the optimization can be applied. It is unlikely that a -relevant occurrence of .* (i.e. one which might indicate anchoring or forcing -the match to follow \n) will appear inside brackets with a number greater than -31, but if it does, any back reference > 31 suppresses the optimization. - -40. Added a new compile-time option PCRE_NO_AUTO_CAPTURE. This has the effect -of disabling numbered capturing parentheses. Any opening parenthesis that is -not followed by ? behaves as if it were followed by ?: but named parentheses -can still be used for capturing (and they will acquire numbers in the usual -way). - -41. Redesigned the return codes from the match() function into yes/no/error so -that errors can be passed back from deep inside the nested calls. A malloc -failure while inside a recursive subpattern call now causes the -PCRE_ERROR_NOMEMORY return instead of quietly going wrong. - -42. It is now possible to set a limit on the number of times the match() -function is called in a call to pcre_exec(). This facility makes it possible to -limit the amount of recursion and backtracking, though not in a directly -obvious way, because the match() function is used in a number of different -circumstances. The count starts from zero for each position in the subject -string (for non-anchored patterns). The default limit is, for compatibility, a -large number, namely 10 000 000. You can change this in two ways: - -(a) When configuring PCRE before making, you can use --with-match-limit=n - to set a default value for the compiled library. - -(b) For each call to pcre_exec(), you can pass a pcre_extra block in which - a different value is set. See 45 below. - -If the limit is exceeded, pcre_exec() returns PCRE_ERROR_MATCHLIMIT. - -43. Added a new function pcre_config(int, void *) to enable run-time extraction -of things that can be changed at compile time. The first argument specifies -what is wanted and the second points to where the information is to be placed. -The current list of available information is: - - PCRE_CONFIG_UTF8 - -The output is an integer that is set to one if UTF-8 support is available; -otherwise it is set to zero. - - PCRE_CONFIG_NEWLINE - -The output is an integer that it set to the value of the code that is used for -newline. It is either LF (10) or CR (13). - - PCRE_CONFIG_LINK_SIZE - -The output is an integer that contains the number of bytes used for internal -linkage in compiled expressions. The value is 2, 3, or 4. See item 32 above. - - PCRE_CONFIG_POSIX_MALLOC_THRESHOLD - -The output is an integer that contains the threshold above which the POSIX -interface uses malloc() for output vectors. See item 31 above. - - PCRE_CONFIG_MATCH_LIMIT - -The output is an unsigned integer that contains the default limit of the number -of match() calls in a pcre_exec() execution. See 42 above. - -44. pcretest has been upgraded by the addition of the -C option. This causes it -to extract all the available output from the new pcre_config() function, and to -output it. The program then exits immediately. - -45. A need has arisen to pass over additional data with calls to pcre_exec() in -order to support additional features. One way would have been to define -pcre_exec2() (for example) with extra arguments, but this would not have been -extensible, and would also have required all calls to the original function to -be mapped to the new one. Instead, I have chosen to extend the mechanism that -is used for passing in "extra" data from pcre_study(). - -The pcre_extra structure is now exposed and defined in pcre.h. It currently -contains the following fields: - - flags a bitmap indicating which of the following fields are set - study_data opaque data from pcre_study() - match_limit a way of specifying a limit on match() calls for a specific - call to pcre_exec() - callout_data data for callouts (see 49 below) - -The flag bits are also defined in pcre.h, and are - - PCRE_EXTRA_STUDY_DATA - PCRE_EXTRA_MATCH_LIMIT - PCRE_EXTRA_CALLOUT_DATA - -The pcre_study() function now returns one of these new pcre_extra blocks, with -the actual study data pointed to by the study_data field, and the -PCRE_EXTRA_STUDY_DATA flag set. This can be passed directly to pcre_exec() as -before. That is, this change is entirely upwards-compatible and requires no -change to existing code. - -If you want to pass in additional data to pcre_exec(), you can either place it -in a pcre_extra block provided by pcre_study(), or create your own pcre_extra -block. - -46. pcretest has been extended to test the PCRE_EXTRA_MATCH_LIMIT feature. If a -data string contains the escape sequence \M, pcretest calls pcre_exec() several -times with different match limits, until it finds the minimum value needed for -pcre_exec() to complete. The value is then output. This can be instructive; for -most simple matches the number is quite small, but for pathological cases it -gets very large very quickly. - -47. There's a new option for pcre_fullinfo() called PCRE_INFO_STUDYSIZE. It -returns the size of the data block pointed to by the study_data field in a -pcre_extra block, that is, the value that was passed as the argument to -pcre_malloc() when PCRE was getting memory in which to place the information -created by pcre_study(). The fourth argument should point to a size_t variable. -pcretest has been extended so that this information is shown after a successful -pcre_study() call when information about the compiled regex is being displayed. - -48. Cosmetic change to Makefile: there's no need to have / after $(DESTDIR) -because what follows is always an absolute path. (Later: it turns out that this -is more than cosmetic for MinGW, because it doesn't like empty path -components.) - -49. Some changes have been made to the callout feature (see 28 above): - -(i) A callout function now has three choices for what it returns: - - 0 => success, carry on matching - > 0 => failure at this point, but backtrack if possible - < 0 => serious error, return this value from pcre_exec() - - Negative values should normally be chosen from the set of PCRE_ERROR_xxx - values. In particular, returning PCRE_ERROR_NOMATCH forces a standard - "match failed" error. The error number PCRE_ERROR_CALLOUT is reserved for - use by callout functions. It will never be used by PCRE itself. - -(ii) The pcre_extra structure (see 45 above) has a void * field called - callout_data, with corresponding flag bit PCRE_EXTRA_CALLOUT_DATA. The - pcre_callout_block structure has a field of the same name. The contents of - the field passed in the pcre_extra structure are passed to the callout - function in the corresponding field in the callout block. This makes it - easier to use the same callout-containing regex from multiple threads. For - testing, the pcretest program has a new data escape - - \C*n pass the number n (may be negative) as callout_data - - If the callout function in pcretest receives a non-zero value as - callout_data, it returns that value. - -50. Makefile wasn't handling CFLAGS properly when compiling dftables. Also, -there were some redundant $(CFLAGS) in commands that are now specified as -$(LINK), which already includes $(CFLAGS). - -51. Extensions to UTF-8 support are listed below. These all apply when (a) PCRE -has been compiled with UTF-8 support *and* pcre_compile() has been compiled -with the PCRE_UTF8 flag. Patterns that are compiled without that flag assume -one-byte characters throughout. Note that case-insensitive matching applies -only to characters whose values are less than 256. PCRE doesn't support the -notion of cases for higher-valued characters. - -(i) A character class whose characters are all within 0-255 is handled as - a bit map, and the map is inverted for negative classes. Previously, a - character > 255 always failed to match such a class; however it should - match if the class was a negative one (e.g. [^ab]). This has been fixed. - -(ii) A negated character class with a single character < 255 is coded as - "not this character" (OP_NOT). This wasn't working properly when the test - character was multibyte, either singly or repeated. - -(iii) Repeats of multibyte characters are now handled correctly in UTF-8 - mode, for example: \x{100}{2,3}. - -(iv) The character escapes \b, \B, \d, \D, \s, \S, \w, and \W (either - singly or repeated) now correctly test multibyte characters. However, - PCRE doesn't recognize any characters with values greater than 255 as - digits, spaces, or word characters. Such characters always match \D, \S, - and \W, and never match \d, \s, or \w. - -(v) Classes may now contain characters and character ranges with values - greater than 255. For example: [ab\x{100}-\x{400}]. - -(vi) pcregrep now has a --utf-8 option (synonym -u) which makes it call - PCRE in UTF-8 mode. - -52. The info request value PCRE_INFO_FIRSTCHAR has been renamed -PCRE_INFO_FIRSTBYTE because it is a byte value. However, the old name is -retained for backwards compatibility. (Note that LASTLITERAL is also a byte -value.) - -53. The single man page has become too large. I have therefore split it up into -a number of separate man pages. These also give rise to individual HTML pages; -these are now put in a separate directory, and there is an index.html page that -lists them all. Some hyperlinking between the pages has been installed. - -54. Added convenience functions for handling named capturing parentheses. - -55. Unknown escapes inside character classes (e.g. [\M]) and escapes that -aren't interpreted therein (e.g. [\C]) are literals in Perl. This is now also -true in PCRE, except when the PCRE_EXTENDED option is set, in which case they -are faulted. - -56. Introduced HOST_CC and HOST_CFLAGS which can be set in the environment when -calling configure. These values are used when compiling the dftables.c program -which is run to generate the source of the default character tables. They -default to the values of CC and CFLAGS. If you are cross-compiling PCRE, -you will need to set these values. - -57. Updated the building process for Windows DLL, as provided by Fred Cox. - - -Version 3.9 02-Jan-02 ---------------------- - -1. A bit of extraneous text had somehow crept into the pcregrep documentation. - -2. If --disable-static was given, the building process failed when trying to -build pcretest and pcregrep. (For some reason it was using libtool to compile -them, which is not right, as they aren't part of the library.) - - -Version 3.8 18-Dec-01 ---------------------- - -1. The experimental UTF-8 code was completely screwed up. It was packing the -bytes in the wrong order. How dumb can you get? - - -Version 3.7 29-Oct-01 ---------------------- - -1. In updating pcretest to check change 1 of version 3.6, I screwed up. -This caused pcretest, when used on the test data, to segfault. Unfortunately, -this didn't happen under Solaris 8, where I normally test things. - -2. The Makefile had to be changed to make it work on BSD systems, where 'make' -doesn't seem to recognize that ./xxx and xxx are the same file. (This entry -isn't in ChangeLog distributed with 3.7 because I forgot when I hastily made -this fix an hour or so after the initial 3.7 release.) - - -Version 3.6 23-Oct-01 ---------------------- - -1. Crashed with /(sens|respons)e and \1ibility/ and "sense and sensibility" if -offsets passed as NULL with zero offset count. - -2. The config.guess and config.sub files had not been updated when I moved to -the latest autoconf. - - -Version 3.5 15-Aug-01 ---------------------- - -1. Added some missing #if !defined NOPOSIX conditionals in pcretest.c that -had been forgotten. - -2. By using declared but undefined structures, we can avoid using "void" -definitions in pcre.h while keeping the internal definitions of the structures -private. - -3. The distribution is now built using autoconf 2.50 and libtool 1.4. From a -user point of view, this means that both static and shared libraries are built -by default, but this can be individually controlled. More of the work of -handling this static/shared cases is now inside libtool instead of PCRE's make -file. - -4. The pcretest utility is now installed along with pcregrep because it is -useful for users (to test regexs) and by doing this, it automatically gets -relinked by libtool. The documentation has been turned into a man page, so -there are now .1, .txt, and .html versions in /doc. - -5. Upgrades to pcregrep: - (i) Added long-form option names like gnu grep. - (ii) Added --help to list all options with an explanatory phrase. - (iii) Added -r, --recursive to recurse into sub-directories. - (iv) Added -f, --file to read patterns from a file. - -6. pcre_exec() was referring to its "code" argument before testing that -argument for NULL (and giving an error if it was NULL). - -7. Upgraded Makefile.in to allow for compiling in a different directory from -the source directory. - -8. Tiny buglet in pcretest: when pcre_fullinfo() was called to retrieve the -options bits, the pointer it was passed was to an int instead of to an unsigned -long int. This mattered only on 64-bit systems. - -9. Fixed typo (3.4/1) in pcre.h again. Sigh. I had changed pcre.h (which is -generated) instead of pcre.in, which it its source. Also made the same change -in several of the .c files. - -10. A new release of gcc defines printf() as a macro, which broke pcretest -because it had an ifdef in the middle of a string argument for printf(). Fixed -by using separate calls to printf(). - -11. Added --enable-newline-is-cr and --enable-newline-is-lf to the configure -script, to force use of CR or LF instead of \n in the source. On non-Unix -systems, the value can be set in config.h. - -12. The limit of 200 on non-capturing parentheses is a _nesting_ limit, not an -absolute limit. Changed the text of the error message to make this clear, and -likewise updated the man page. - -13. The limit of 99 on the number of capturing subpatterns has been removed. -The new limit is 65535, which I hope will not be a "real" limit. - - -Version 3.4 22-Aug-00 ---------------------- - -1. Fixed typo in pcre.h: unsigned const char * changed to const unsigned char *. - -2. Diagnose condition (?(0) as an error instead of crashing on matching. - - -Version 3.3 01-Aug-00 ---------------------- - -1. If an octal character was given, but the value was greater than \377, it -was not getting masked to the least significant bits, as documented. This could -lead to crashes in some systems. - -2. Perl 5.6 (if not earlier versions) accepts classes like [a-\d] and treats -the hyphen as a literal. PCRE used to give an error; it now behaves like Perl. - -3. Added the functions pcre_free_substring() and pcre_free_substring_list(). -These just pass their arguments on to (pcre_free)(), but they are provided -because some uses of PCRE bind it to non-C systems that can call its functions, -but cannot call free() or pcre_free() directly. - -4. Add "make test" as a synonym for "make check". Corrected some comments in -the Makefile. - -5. Add $(DESTDIR)/ in front of all the paths in the "install" target in the -Makefile. - -6. Changed the name of pgrep to pcregrep, because Solaris has introduced a -command called pgrep for grepping around the active processes. - -7. Added the beginnings of support for UTF-8 character strings. - -8. Arranged for the Makefile to pass over the settings of CC, CFLAGS, and -RANLIB to ./ltconfig so that they are used by libtool. I think these are all -the relevant ones. (AR is not passed because ./ltconfig does its own figuring -out for the ar command.) - - -Version 3.2 12-May-00 ---------------------- - -This is purely a bug fixing release. - -1. If the pattern /((Z)+|A)*/ was matched agained ZABCDEFG it matched Z instead -of ZA. This was just one example of several cases that could provoke this bug, -which was introduced by change 9 of version 2.00. The code for breaking -infinite loops after an iteration that matches an empty string was't working -correctly. - -2. The pcretest program was not imitating Perl correctly for the pattern /a*/g -when matched against abbab (for example). After matching an empty string, it -wasn't forcing anchoring when setting PCRE_NOTEMPTY for the next attempt; this -caused it to match further down the string than it should. - -3. The code contained an inclusion of sys/types.h. It isn't clear why this -was there because it doesn't seem to be needed, and it causes trouble on some -systems, as it is not a Standard C header. It has been removed. - -4. Made 4 silly changes to the source to avoid stupid compiler warnings that -were reported on the Macintosh. The changes were from - - while ((c = *(++ptr)) != 0 && c != '\n'); -to - while ((c = *(++ptr)) != 0 && c != '\n') ; - -Totally extraordinary, but if that's what it takes... - -5. PCRE is being used in one environment where neither memmove() nor bcopy() is -available. Added HAVE_BCOPY and an autoconf test for it; if neither -HAVE_MEMMOVE nor HAVE_BCOPY is set, use a built-in emulation function which -assumes the way PCRE uses memmove() (always moving upwards). - -6. PCRE is being used in one environment where strchr() is not available. There -was only one use in pcre.c, and writing it out to avoid strchr() probably gives -faster code anyway. - - -Version 3.1 09-Feb-00 ---------------------- - -The only change in this release is the fixing of some bugs in Makefile.in for -the "install" target: - -(1) It was failing to install pcreposix.h. - -(2) It was overwriting the pcre.3 man page with the pcreposix.3 man page. - - -Version 3.0 01-Feb-00 ---------------------- - -1. Add support for the /+ modifier to perltest (to output $` like it does in -pcretest). - -2. Add support for the /g modifier to perltest. - -3. Fix pcretest so that it behaves even more like Perl for /g when the pattern -matches null strings. - -4. Fix perltest so that it doesn't do unwanted things when fed an empty -pattern. Perl treats empty patterns specially - it reuses the most recent -pattern, which is not what we want. Replace // by /(?#)/ in order to avoid this -effect. - -5. The POSIX interface was broken in that it was just handing over the POSIX -captured string vector to pcre_exec(), but (since release 2.00) PCRE has -required a bigger vector, with some working space on the end. This means that -the POSIX wrapper now has to get and free some memory, and copy the results. - -6. Added some simple autoconf support, placing the test data and the -documentation in separate directories, re-organizing some of the -information files, and making it build pcre-config (a GNU standard). Also added -libtool support for building PCRE as a shared library, which is now the -default. - -7. Got rid of the leading zero in the definition of PCRE_MINOR because 08 and -09 are not valid octal constants. Single digits will be used for minor values -less than 10. - -8. Defined REG_EXTENDED and REG_NOSUB as zero in the POSIX header, so that -existing programs that set these in the POSIX interface can use PCRE without -modification. - -9. Added a new function, pcre_fullinfo() with an extensible interface. It can -return all that pcre_info() returns, plus additional data. The pcre_info() -function is retained for compatibility, but is considered to be obsolete. - -10. Added experimental recursion feature (?R) to handle one common case that -Perl 5.6 will be able to do with (?p{...}). - -11. Added support for POSIX character classes like [:alpha:], which Perl is -adopting. - - -Version 2.08 31-Aug-99 ----------------------- - -1. When startoffset was not zero and the pattern began with ".*", PCRE was not -trying to match at the startoffset position, but instead was moving forward to -the next newline as if a previous match had failed. - -2. pcretest was not making use of PCRE_NOTEMPTY when repeating for /g and /G, -and could get into a loop if a null string was matched other than at the start -of the subject. - -3. Added definitions of PCRE_MAJOR and PCRE_MINOR to pcre.h so the version can -be distinguished at compile time, and for completeness also added PCRE_DATE. - -5. Added Paul Sokolovsky's minor changes to make it easy to compile a Win32 DLL -in GnuWin32 environments. - - -Version 2.07 29-Jul-99 ----------------------- - -1. The documentation is now supplied in plain text form and HTML as well as in -the form of man page sources. - -2. C++ compilers don't like assigning (void *) values to other pointer types. -In particular this affects malloc(). Although there is no problem in Standard -C, I've put in casts to keep C++ compilers happy. - -3. Typo on pcretest.c; a cast of (unsigned char *) in the POSIX regexec() call -should be (const char *). - -4. If NOPOSIX is defined, pcretest.c compiles without POSIX support. This may -be useful for non-Unix systems who don't want to bother with the POSIX stuff. -However, I haven't made this a standard facility. The documentation doesn't -mention it, and the Makefile doesn't support it. - -5. The Makefile now contains an "install" target, with editable destinations at -the top of the file. The pcretest program is not installed. - -6. pgrep -V now gives the PCRE version number and date. - -7. Fixed bug: a zero repetition after a literal string (e.g. /abcde{0}/) was -causing the entire string to be ignored, instead of just the last character. - -8. If a pattern like /"([^\\"]+|\\.)*"/ is applied in the normal way to a -non-matching string, it can take a very, very long time, even for strings of -quite modest length, because of the nested recursion. PCRE now does better in -some of these cases. It does this by remembering the last required literal -character in the pattern, and pre-searching the subject to ensure it is present -before running the real match. In other words, it applies a heuristic to detect -some types of certain failure quickly, and in the above example, if presented -with a string that has no trailing " it gives "no match" very quickly. - -9. A new runtime option PCRE_NOTEMPTY causes null string matches to be ignored; -other alternatives are tried instead. - - -Version 2.06 09-Jun-99 ----------------------- - -1. Change pcretest's output for amount of store used to show just the code -space, because the remainder (the data block) varies in size between 32-bit and -64-bit systems. - -2. Added an extra argument to pcre_exec() to supply an offset in the subject to -start matching at. This allows lookbehinds to work when searching for multiple -occurrences in a string. - -3. Added additional options to pcretest for testing multiple occurrences: - - /+ outputs the rest of the string that follows a match - /g loops for multiple occurrences, using the new startoffset argument - /G loops for multiple occurrences by passing an incremented pointer - -4. PCRE wasn't doing the "first character" optimization for patterns starting -with \b or \B, though it was doing it for other lookbehind assertions. That is, -it wasn't noticing that a match for a pattern such as /\bxyz/ has to start with -the letter 'x'. On long subject strings, this gives a significant speed-up. - - -Version 2.05 21-Apr-99 ----------------------- - -1. Changed the type of magic_number from int to long int so that it works -properly on 16-bit systems. - -2. Fixed a bug which caused patterns starting with .* not to work correctly -when the subject string contained newline characters. PCRE was assuming -anchoring for such patterns in all cases, which is not correct because .* will -not pass a newline unless PCRE_DOTALL is set. It now assumes anchoring only if -DOTALL is set at top level; otherwise it knows that patterns starting with .* -must be retried after every newline in the subject. - - -Version 2.04 18-Feb-99 ----------------------- - -1. For parenthesized subpatterns with repeats whose minimum was zero, the -computation of the store needed to hold the pattern was incorrect (too large). -If such patterns were nested a few deep, this could multiply and become a real -problem. - -2. Added /M option to pcretest to show the memory requirement of a specific -pattern. Made -m a synonym of -s (which does this globally) for compatibility. - -3. Subpatterns of the form (regex){n,m} (i.e. limited maximum) were being -compiled in such a way that the backtracking after subsequent failure was -pessimal. Something like (a){0,3} was compiled as (a)?(a)?(a)? instead of -((a)((a)(a)?)?)? with disastrous performance if the maximum was of any size. - - -Version 2.03 02-Feb-99 ----------------------- - -1. Fixed typo and small mistake in man page. - -2. Added 4th condition (GPL supersedes if conflict) and created separate -LICENCE file containing the conditions. - -3. Updated pcretest so that patterns such as /abc\/def/ work like they do in -Perl, that is the internal \ allows the delimiter to be included in the -pattern. Locked out the use of \ as a delimiter. If \ immediately follows -the final delimiter, add \ to the end of the pattern (to test the error). - -4. Added the convenience functions for extracting substrings after a successful -match. Updated pcretest to make it able to test these functions. - - -Version 2.02 14-Jan-99 ----------------------- - -1. Initialized the working variables associated with each extraction so that -their saving and restoring doesn't refer to uninitialized store. - -2. Put dummy code into study.c in order to trick the optimizer of the IBM C -compiler for OS/2 into generating correct code. Apparently IBM isn't going to -fix the problem. - -3. Pcretest: the timing code wasn't using LOOPREPEAT for timing execution -calls, and wasn't printing the correct value for compiling calls. Increased the -default value of LOOPREPEAT, and the number of significant figures in the -times. - -4. Changed "/bin/rm" in the Makefile to "-rm" so it works on Windows NT. - -5. Renamed "deftables" as "dftables" to get it down to 8 characters, to avoid -a building problem on Windows NT with a FAT file system. - - -Version 2.01 21-Oct-98 ----------------------- - -1. Changed the API for pcre_compile() to allow for the provision of a pointer -to character tables built by pcre_maketables() in the current locale. If NULL -is passed, the default tables are used. - - -Version 2.00 24-Sep-98 ----------------------- - -1. Since the (>?) facility is in Perl 5.005, don't require PCRE_EXTRA to enable -it any more. - -2. Allow quantification of (?>) groups, and make it work correctly. - -3. The first character computation wasn't working for (?>) groups. - -4. Correct the implementation of \Z (it is permitted to match on the \n at the -end of the subject) and add 5.005's \z, which really does match only at the -very end of the subject. - -5. Remove the \X "cut" facility; Perl doesn't have it, and (?> is neater. - -6. Remove the ability to specify CASELESS, MULTILINE, DOTALL, and -DOLLAR_END_ONLY at runtime, to make it possible to implement the Perl 5.005 -localized options. All options to pcre_study() were also removed. - -7. Add other new features from 5.005: - - $(?<= positive lookbehind - $(?<! negative lookbehind - (?imsx-imsx) added the unsetting capability - such a setting is global if at outer level; local otherwise - (?imsx-imsx:) non-capturing groups with option setting - (?(cond)re|re) conditional pattern matching - - A backreference to itself in a repeated group matches the previous - captured string. - -8. General tidying up of studying (both automatic and via "study") -consequential on the addition of new assertions. - -9. As in 5.005, unlimited repeated groups that could match an empty substring -are no longer faulted at compile time. Instead, the loop is forcibly broken at -runtime if any iteration does actually match an empty substring. - -10. Include the RunTest script in the distribution. - -11. Added tests from the Perl 5.005_02 distribution. This showed up a few -discrepancies, some of which were old and were also with respect to 5.004. They -have now been fixed. - - -Version 1.09 28-Apr-98 ----------------------- - -1. A negated single character class followed by a quantifier with a minimum -value of one (e.g. [^x]{1,6} ) was not compiled correctly. This could lead to -program crashes, or just wrong answers. This did not apply to negated classes -containing more than one character, or to minima other than one. - - -Version 1.08 27-Mar-98 ----------------------- - -1. Add PCRE_UNGREEDY to invert the greediness of quantifiers. - -2. Add (?U) and (?X) to set PCRE_UNGREEDY and PCRE_EXTRA respectively. The -latter must appear before anything that relies on it in the pattern. - - -Version 1.07 16-Feb-98 ----------------------- - -1. A pattern such as /((a)*)*/ was not being diagnosed as in error (unlimited -repeat of a potentially empty string). - - -Version 1.06 23-Jan-98 ----------------------- - -1. Added Markus Oberhumer's little patches for C++. - -2. Literal strings longer than 255 characters were broken. - - -Version 1.05 23-Dec-97 ----------------------- - -1. Negated character classes containing more than one character were failing if -PCRE_CASELESS was set at run time. - - -Version 1.04 19-Dec-97 ----------------------- - -1. Corrected the man page, where some "const" qualifiers had been omitted. - -2. Made debugging output print "{0,xxx}" instead of just "{,xxx}" to agree with -input syntax. - -3. Fixed memory leak which occurred when a regex with back references was -matched with an offsets vector that wasn't big enough. The temporary memory -that is used in this case wasn't being freed if the match failed. - -4. Tidied pcretest to ensure it frees memory that it gets. - -5. Temporary memory was being obtained in the case where the passed offsets -vector was exactly big enough. - -6. Corrected definition of offsetof() from change 5 below. - -7. I had screwed up change 6 below and broken the rules for the use of -setjmp(). Now fixed. - - -Version 1.03 18-Dec-97 ----------------------- - -1. A erroneous regex with a missing opening parenthesis was correctly -diagnosed, but PCRE attempted to access brastack[-1], which could cause crashes -on some systems. - -2. Replaced offsetof(real_pcre, code) by offsetof(real_pcre, code[0]) because -it was reported that one broken compiler failed on the former because "code" is -also an independent variable. - -3. The erroneous regex a[]b caused an array overrun reference. - -4. A regex ending with a one-character negative class (e.g. /[^k]$/) did not -fail on data ending with that character. (It was going on too far, and checking -the next character, typically a binary zero.) This was specific to the -optimized code for single-character negative classes. - -5. Added a contributed patch from the TIN world which does the following: - - + Add an undef for memmove, in case the the system defines a macro for it. - - + Add a definition of offsetof(), in case there isn't one. (I don't know - the reason behind this - offsetof() is part of the ANSI standard - but - it does no harm). - - + Reduce the ifdef's in pcre.c using macro DPRINTF, thereby eliminating - most of the places where whitespace preceded '#'. I have given up and - allowed the remaining 2 cases to be at the margin. - - + Rename some variables in pcre to eliminate shadowing. This seems very - pedantic, but does no harm, of course. - -6. Moved the call to setjmp() into its own function, to get rid of warnings -from gcc -Wall, and avoided calling it at all unless PCRE_EXTRA is used. - -7. Constructs such as \d{8,} were compiling into the equivalent of -\d{8}\d{0,65527} instead of \d{8}\d* which didn't make much difference to the -outcome, but in this particular case used more store than had been allocated, -which caused the bug to be discovered because it threw up an internal error. - -8. The debugging code in both pcre and pcretest for outputting the compiled -form of a regex was going wrong in the case of back references followed by -curly-bracketed repeats. - - -Version 1.02 12-Dec-97 ----------------------- - -1. Typos in pcre.3 and comments in the source fixed. - -2. Applied a contributed patch to get rid of places where it used to remove -'const' from variables, and fixed some signed/unsigned and uninitialized -variable warnings. - -3. Added the "runtest" target to Makefile. - -4. Set default compiler flag to -O2 rather than just -O. - - -Version 1.01 19-Nov-97 ----------------------- - -1. PCRE was failing to diagnose unlimited repeat of empty string for patterns -like /([ab]*)*/, that is, for classes with more than one character in them. - -2. Likewise, it wasn't diagnosing patterns with "once-only" subpatterns, such -as /((?>a*))*/ (a PCRE_EXTRA facility). - - -Version 1.00 18-Nov-97 ----------------------- - -1. Added compile-time macros to support systems such as SunOS4 which don't have -memmove() or strerror() but have other things that can be used instead. - -2. Arranged that "make clean" removes the executables. - - -Version 0.99 27-Oct-97 ----------------------- - -1. Fixed bug in code for optimizing classes with only one character. It was -initializing a 32-byte map regardless, which could cause it to run off the end -of the memory it had got. - -2. Added, conditional on PCRE_EXTRA, the proposed (?>REGEX) construction. - - -Version 0.98 22-Oct-97 ----------------------- - -1. Fixed bug in code for handling temporary memory usage when there are more -back references than supplied space in the ovector. This could cause segfaults. - - -Version 0.97 21-Oct-97 ----------------------- - -1. Added the \X "cut" facility, conditional on PCRE_EXTRA. - -2. Optimized negated single characters not to use a bit map. - -3. Brought error texts together as macro definitions; clarified some of them; -fixed one that was wrong - it said "range out of order" when it meant "invalid -escape sequence". - -4. Changed some char * arguments to const char *. - -5. Added PCRE_NOTBOL and PCRE_NOTEOL (from POSIX). - -6. Added the POSIX-style API wrapper in pcreposix.a and testing facilities in -pcretest. - - -Version 0.96 16-Oct-97 ----------------------- - -1. Added a simple "pgrep" utility to the distribution. - -2. Fixed an incompatibility with Perl: "{" is now treated as a normal character -unless it appears in one of the precise forms "{ddd}", "{ddd,}", or "{ddd,ddd}" -where "ddd" means "one or more decimal digits". - -3. Fixed serious bug. If a pattern had a back reference, but the call to -pcre_exec() didn't supply a large enough ovector to record the related -identifying subpattern, the match always failed. PCRE now remembers the number -of the largest back reference, and gets some temporary memory in which to save -the offsets during matching if necessary, in order to ensure that -backreferences always work. - -4. Increased the compatibility with Perl in a number of ways: - - (a) . no longer matches \n by default; an option PCRE_DOTALL is provided - to request this handling. The option can be set at compile or exec time. - - (b) $ matches before a terminating newline by default; an option - PCRE_DOLLAR_ENDONLY is provided to override this (but not in multiline - mode). The option can be set at compile or exec time. - - (c) The handling of \ followed by a digit other than 0 is now supposed to be - the same as Perl's. If the decimal number it represents is less than 10 - or there aren't that many previous left capturing parentheses, an octal - escape is read. Inside a character class, it's always an octal escape, - even if it is a single digit. - - (d) An escaped but undefined alphabetic character is taken as a literal, - unless PCRE_EXTRA is set. Currently this just reserves the remaining - escapes. - - (e) {0} is now permitted. (The previous item is removed from the compiled - pattern). - -5. Changed all the names of code files so that the basic parts are no longer -than 10 characters, and abolished the teeny "globals.c" file. - -6. Changed the handling of character classes; they are now done with a 32-byte -bit map always. - -7. Added the -d and /D options to pcretest to make it possible to look at the -internals of compilation without having to recompile pcre. - - -Version 0.95 23-Sep-97 ----------------------- - -1. Fixed bug in pre-pass concerning escaped "normal" characters such as \x5c or -\x20 at the start of a run of normal characters. These were being treated as -real characters, instead of the source characters being re-checked. - - -Version 0.94 18-Sep-97 ----------------------- - -1. The functions are now thread-safe, with the caveat that the global variables -containing pointers to malloc() and free() or alternative functions are the -same for all threads. - -2. Get pcre_study() to generate a bitmap of initial characters for non- -anchored patterns when this is possible, and use it if passed to pcre_exec(). - - -Version 0.93 15-Sep-97 ----------------------- - -1. /(b)|(:+)/ was computing an incorrect first character. - -2. Add pcre_study() to the API and the passing of pcre_extra to pcre_exec(), -but not actually doing anything yet. - -3. Treat "-" characters in classes that cannot be part of ranges as literals, -as Perl does (e.g. [-az] or [az-]). - -4. Set the anchored flag if a branch starts with .* or .*? because that tests -all possible positions. - -5. Split up into different modules to avoid including unneeded functions in a -compiled binary. However, compile and exec are still in one module. The "study" -function is split off. - -6. The character tables are now in a separate module whose source is generated -by an auxiliary program - but can then be edited by hand if required. There are -now no calls to isalnum(), isspace(), isdigit(), isxdigit(), tolower() or -toupper() in the code. - -7. Turn the malloc/free funtions variables into pcre_malloc and pcre_free and -make them global. Abolish the function for setting them, as the caller can now -set them directly. - - -Version 0.92 11-Sep-97 ----------------------- - -1. A repeat with a fixed maximum and a minimum of 1 for an ordinary character -(e.g. /a{1,3}/) was broken (I mis-optimized it). - -2. Caseless matching was not working in character classes if the characters in -the pattern were in upper case. - -3. Make ranges like [W-c] work in the same way as Perl for caseless matching. - -4. Make PCRE_ANCHORED public and accept as a compile option. - -5. Add an options word to pcre_exec() and accept PCRE_ANCHORED and -PCRE_CASELESS at run time. Add escapes \A and \I to pcretest to cause it to -pass them. - -6. Give an error if bad option bits passed at compile or run time. - -7. Add PCRE_MULTILINE at compile and exec time, and (?m) as well. Add \M to -pcretest to cause it to pass that flag. - -8. Add pcre_info(), to get the number of identifying subpatterns, the stored -options, and the first character, if set. - -9. Recognize C+ or C{n,m} where n >= 1 as providing a fixed starting character. - - -Version 0.91 10-Sep-97 ----------------------- - -1. PCRE was failing to diagnose unlimited repeats of subpatterns that could -match the empty string as in /(a*)*/. It was looping and ultimately crashing. - -2. PCRE was looping on encountering an indefinitely repeated back reference to -a subpattern that had matched an empty string, e.g. /(a|)\1*/. It now does what -Perl does - treats the match as successful. - -**** diff --git a/plugins/Pcre16/docs/HACKING b/plugins/Pcre16/docs/HACKING deleted file mode 100644 index 691b7a14e5..0000000000 --- a/plugins/Pcre16/docs/HACKING +++ /dev/null @@ -1,528 +0,0 @@ -Technical Notes about PCRE --------------------------- - -These are very rough technical notes that record potentially useful information -about PCRE internals. For information about testing PCRE, see the pcretest -documentation and the comment at the head of the RunTest file. - - -Historical note 1 ------------------ - -Many years ago I implemented some regular expression functions to an algorithm -suggested by Martin Richards. These were not Unix-like in form, and were quite -restricted in what they could do by comparison with Perl. The interesting part -about the algorithm was that the amount of space required to hold the compiled -form of an expression was known in advance. The code to apply an expression did -not operate by backtracking, as the original Henry Spencer code and current -Perl code does, but instead checked all possibilities simultaneously by keeping -a list of current states and checking all of them as it advanced through the -subject string. In the terminology of Jeffrey Friedl's book, it was a "DFA -algorithm", though it was not a traditional Finite State Machine (FSM). When -the pattern was all used up, all remaining states were possible matches, and -the one matching the longest subset of the subject string was chosen. This did -not necessarily maximize the individual wild portions of the pattern, as is -expected in Unix and Perl-style regular expressions. - - -Historical note 2 ------------------ - -By contrast, the code originally written by Henry Spencer (which was -subsequently heavily modified for Perl) compiles the expression twice: once in -a dummy mode in order to find out how much store will be needed, and then for -real. (The Perl version probably doesn't do this any more; I'm talking about -the original library.) The execution function operates by backtracking and -maximizing (or, optionally, minimizing in Perl) the amount of the subject that -matches individual wild portions of the pattern. This is an "NFA algorithm" in -Friedl's terminology. - - -OK, here's the real stuff -------------------------- - -For the set of functions that form the "basic" PCRE library (which are -unrelated to those mentioned above), I tried at first to invent an algorithm -that used an amount of store bounded by a multiple of the number of characters -in the pattern, to save on compiling time. However, because of the greater -complexity in Perl regular expressions, I couldn't do this. In any case, a -first pass through the pattern is helpful for other reasons. - - -Support for 16-bit and 32-bit data strings -------------------------------------------- - -From release 8.30, PCRE supports 16-bit as well as 8-bit data strings; and from -release 8.32, PCRE supports 32-bit data strings. The library can be compiled -in any combination of 8-bit, 16-bit or 32-bit modes, creating up to three -different libraries. In the description that follows, the word "short" is used -for a 16-bit data quantity, and the word "unit" is used for a quantity that is -a byte in 8-bit mode, a short in 16-bit mode and a 32-bit word in 32-bit mode. -However, so as not to over-complicate the text, the names of PCRE functions are -given in 8-bit form only. - - -Computing the memory requirement: how it was --------------------------------------------- - -Up to and including release 6.7, PCRE worked by running a very degenerate first -pass to calculate a maximum store size, and then a second pass to do the real -compile - which might use a bit less than the predicted amount of memory. The -idea was that this would turn out faster than the Henry Spencer code because -the first pass is degenerate and the second pass can just store stuff straight -into the vector, which it knows is big enough. - - -Computing the memory requirement: how it is -------------------------------------------- - -By the time I was working on a potential 6.8 release, the degenerate first pass -had become very complicated and hard to maintain. Indeed one of the early -things I did for 6.8 was to fix Yet Another Bug in the memory computation. Then -I had a flash of inspiration as to how I could run the real compile function in -a "fake" mode that enables it to compute how much memory it would need, while -actually only ever using a few hundred bytes of working memory, and without too -many tests of the mode that might slow it down. So I refactored the compiling -functions to work this way. This got rid of about 600 lines of source. It -should make future maintenance and development easier. As this was such a major -change, I never released 6.8, instead upping the number to 7.0 (other quite -major changes were also present in the 7.0 release). - -A side effect of this work was that the previous limit of 200 on the nesting -depth of parentheses was removed. However, there is a downside: pcre_compile() -runs more slowly than before (30% or more, depending on the pattern) because it -is doing a full analysis of the pattern. My hope was that this would not be a -big issue, and in the event, nobody has commented on it. - -At release 8.34, a limit on the nesting depth of parentheses was re-introduced -(default 250, settable at build time) so as to put a limit on the amount of -system stack used by pcre_compile(). This is a safety feature for environments -with small stacks where the patterns are provided by users. - - -Traditional matching function ------------------------------ - -The "traditional", and original, matching function is called pcre_exec(), and -it implements an NFA algorithm, similar to the original Henry Spencer algorithm -and the way that Perl works. This is not surprising, since it is intended to be -as compatible with Perl as possible. This is the function most users of PCRE -will use most of the time. From release 8.20, if PCRE is compiled with -just-in-time (JIT) support, and studying a compiled pattern with JIT is -successful, the JIT code is run instead of the normal pcre_exec() code, but the -result is the same. - - -Supplementary matching function -------------------------------- - -From PCRE 6.0, there is also a supplementary matching function called -pcre_dfa_exec(). This implements a DFA matching algorithm that searches -simultaneously for all possible matches that start at one point in the subject -string. (Going back to my roots: see Historical Note 1 above.) This function -intreprets the same compiled pattern data as pcre_exec(); however, not all the -facilities are available, and those that are do not always work in quite the -same way. See the user documentation for details. - -The algorithm that is used for pcre_dfa_exec() is not a traditional FSM, -because it may have a number of states active at one time. More work would be -needed at compile time to produce a traditional FSM where only one state is -ever active at once. I believe some other regex matchers work this way. JIT -support is not available for this kind of matching. - - -Changeable options ------------------- - -The /i, /m, or /s options (PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and some -others) may change in the middle of patterns. From PCRE 8.13, their processing -is handled entirely at compile time by generating different opcodes for the -different settings. The runtime functions do not need to keep track of an -options state any more. - - -Format of compiled patterns ---------------------------- - -The compiled form of a pattern is a vector of unsigned units (bytes in 8-bit -mode, shorts in 16-bit mode, 32-bit words in 32-bit mode), containing items of -variable length. The first unit in an item contains an opcode, and the length -of the item is either implicit in the opcode or contained in the data that -follows it. - -In many cases listed below, LINK_SIZE data values are specified for offsets -within the compiled pattern. LINK_SIZE always specifies a number of bytes. The -default value for LINK_SIZE is 2, but PCRE can be compiled to use 3-byte or -4-byte values for these offsets, although this impairs the performance. (3-byte -LINK_SIZE values are available only in 8-bit mode.) Specifing a LINK_SIZE -larger than 2 is necessary only when patterns whose compiled length is greater -than 64K are going to be processed. In this description, we assume the "normal" -compilation options. Data values that are counts (e.g. quantifiers) are two -bytes long in 8-bit mode (most significant byte first), or one unit in 16-bit -and 32-bit modes. - - -Opcodes with no following data ------------------------------- - -These items are all just one unit long - - OP_END end of pattern - OP_ANY match any one character other than newline - OP_ALLANY match any one character, including newline - OP_ANYBYTE match any single unit, even in UTF-8/16 mode - OP_SOD match start of data: \A - OP_SOM, start of match (subject + offset): \G - OP_SET_SOM, set start of match (\K) - OP_CIRC ^ (start of data) - OP_CIRCM ^ multiline mode (start of data or after newline) - OP_NOT_WORD_BOUNDARY \W - OP_WORD_BOUNDARY \w - OP_NOT_DIGIT \D - OP_DIGIT \d - OP_NOT_HSPACE \H - OP_HSPACE \h - OP_NOT_WHITESPACE \S - OP_WHITESPACE \s - OP_NOT_VSPACE \V - OP_VSPACE \v - OP_NOT_WORDCHAR \W - OP_WORDCHAR \w - OP_EODN match end of data or newline at end: \Z - OP_EOD match end of data: \z - OP_DOLL $ (end of data, or before final newline) - OP_DOLLM $ multiline mode (end of data or before newline) - OP_EXTUNI match an extended Unicode grapheme cluster - OP_ANYNL match any Unicode newline sequence - - OP_ASSERT_ACCEPT ) - OP_ACCEPT ) These are Perl 5.10's "backtracking control - OP_COMMIT ) verbs". If OP_ACCEPT is inside capturing - OP_FAIL ) parentheses, it may be preceded by one or more - OP_PRUNE ) OP_CLOSE, each followed by a count that - OP_SKIP ) indicates which parentheses must be closed. - OP_THEN ) - -OP_ASSERT_ACCEPT is used when (*ACCEPT) is encountered within an assertion. -This ends the assertion, not the entire pattern match. - - -Backtracking control verbs with optional data ---------------------------------------------- - -(*THEN) without an argument generates the opcode OP_THEN and no following data. -OP_MARK is followed by the mark name, preceded by a one-unit length, and -followed by a binary zero. For (*PRUNE), (*SKIP), and (*THEN) with arguments, -the opcodes OP_PRUNE_ARG, OP_SKIP_ARG, and OP_THEN_ARG are used, with the name -following in the same format as OP_MARK. - - -Matching literal characters ---------------------------- - -The OP_CHAR opcode is followed by a single character that is to be matched -casefully. For caseless matching, OP_CHARI is used. In UTF-8 or UTF-16 modes, -the character may be more than one unit long. In UTF-32 mode, characters -are always exactly one unit long. - -If there is only one character in a character class, OP_CHAR or OP_CHARI is -used for a positive class, and OP_NOT or OP_NOTI for a negative one (that is, -for something like [^a]). - - -Repeating single characters ---------------------------- - -The common repeats (*, +, ?), when applied to a single character, use the -following opcodes, which come in caseful and caseless versions: - - Caseful Caseless - OP_STAR OP_STARI - OP_MINSTAR OP_MINSTARI - OP_POSSTAR OP_POSSTARI - OP_PLUS OP_PLUSI - OP_MINPLUS OP_MINPLUSI - OP_POSPLUS OP_POSPLUSI - OP_QUERY OP_QUERYI - OP_MINQUERY OP_MINQUERYI - OP_POSQUERY OP_POSQUERYI - -Each opcode is followed by the character that is to be repeated. In ASCII mode, -these are two-unit items; in UTF-8 or UTF-16 modes, the length is variable; in -UTF-32 mode these are one-unit items. Those with "MIN" in their names are the -minimizing versions. Those with "POS" in their names are possessive versions. -Other repeats make use of these opcodes: - - Caseful Caseless - OP_UPTO OP_UPTOI - OP_MINUPTO OP_MINUPTOI - OP_POSUPTO OP_POSUPTOI - OP_EXACT OP_EXACTI - -Each of these is followed by a count and then the repeated character. OP_UPTO -matches from 0 to the given number. A repeat with a non-zero minimum and a -fixed maximum is coded as an OP_EXACT followed by an OP_UPTO (or OP_MINUPTO or -OPT_POSUPTO). - -Another set of matching repeating opcodes (called OP_NOTSTAR, OP_NOTSTARI, -etc.) are used for repeated, negated, single-character classes such as [^a]*. -The normal single-character opcodes (OP_STAR, etc.) are used for repeated -positive single-character classes. - - -Repeating character types -------------------------- - -Repeats of things like \d are done exactly as for single characters, except -that instead of a character, the opcode for the type is stored in the data -unit. The opcodes are: - - OP_TYPESTAR - OP_TYPEMINSTAR - OP_TYPEPOSSTAR - OP_TYPEPLUS - OP_TYPEMINPLUS - OP_TYPEPOSPLUS - OP_TYPEQUERY - OP_TYPEMINQUERY - OP_TYPEPOSQUERY - OP_TYPEUPTO - OP_TYPEMINUPTO - OP_TYPEPOSUPTO - OP_TYPEEXACT - - -Match by Unicode property -------------------------- - -OP_PROP and OP_NOTPROP are used for positive and negative matches of a -character by testing its Unicode property (the \p and \P escape sequences). -Each is followed by two units that encode the desired property as a type and a -value. The types are a set of #defines of the form PT_xxx, and the values are -enumerations of the form ucp_xx, defined in the ucp.h source file. The value is -relevant only for PT_GC (General Category), PT_PC (Particular Category), and -PT_SC (Script). - -Repeats of these items use the OP_TYPESTAR etc. set of opcodes, followed by -three units: OP_PROP or OP_NOTPROP, and then the desired property type and -value. - - -Character classes ------------------ - -If there is only one character in a class, OP_CHAR or OP_CHARI is used for a -positive class, and OP_NOT or OP_NOTI for a negative one (that is, for -something like [^a]). - -A set of repeating opcodes (called OP_NOTSTAR etc.) are used for repeated, -negated, single-character classes. The normal single-character opcodes -(OP_STAR, etc.) are used for repeated positive single-character classes. - -When there is more than one character in a class, and all the code points are -less than 256, OP_CLASS is used for a positive class, and OP_NCLASS for a -negative one. In either case, the opcode is followed by a 32-byte (16-short, -8-word) bit map containing a 1 bit for every character that is acceptable. The -bits are counted from the least significant end of each unit. In caseless mode, -bits for both cases are set. - -The reason for having both OP_CLASS and OP_NCLASS is so that, in UTF-8/16/32 -mode, subject characters with values greater than 255 can be handled correctly. -For OP_CLASS they do not match, whereas for OP_NCLASS they do. - -For classes containing characters with values greater than 255 or that contain -\p or \P, OP_XCLASS is used. It optionally uses a bit map if any code points -are less than 256, followed by a list of pairs (for a range) and single -characters. In caseless mode, both cases are explicitly listed. - -OP_XCLASS is followed by a unit containing flag bits: XCL_NOT indicates that -this is a negative class, and XCL_MAP indicates that a bit map is present. -There follows the bit map, if XCL_MAP is set, and then a sequence of items -coded as follows: - - XCL_END marks the end of the list - XCL_SINGLE one character follows - XCL_RANGE two characters follow - XCL_PROP a Unicode property (type, value) follows - XCL_NOTPROP a Unicode property (type, value) follows - -If a range starts with a code point less than 256 and ends with one greater -than 256, an XCL_RANGE item is used, without setting any bits in the bit map. -This means that if no other items in the class set bits in the map, a map is -not needed. - - -Back references ---------------- - -OP_REF (caseful) or OP_REFI (caseless) is followed by a count containing the -reference number if the reference is to a unique capturing group (either by -number or by name). When named groups are used, there may be more than one -group with the same name. In this case, a reference by name generates OP_DNREF -or OP_DNREFI. These are followed by two counts: the index (not the byte offset) -in the group name table of the first entry for the requred name, followed by -the number of groups with the same name. - - -Repeating character classes and back references ------------------------------------------------ - -Single-character classes are handled specially (see above). This section -applies to other classes and also to back references. In both cases, the repeat -information follows the base item. The matching code looks at the following -opcode to see if it is one of - - OP_CRSTAR - OP_CRMINSTAR - OP_CRPOSSTAR - OP_CRPLUS - OP_CRMINPLUS - OP_CRPOSPLUS - OP_CRQUERY - OP_CRMINQUERY - OP_CRPOSQUERY - OP_CRRANGE - OP_CRMINRANGE - OP_CRPOSRANGE - -All but the last three are single-unit items, with no data. The others are -followed by the minimum and maximum repeat counts. - - -Brackets and alternation ------------------------- - -A pair of non-capturing round brackets is wrapped round each expression at -compile time, so alternation always happens in the context of brackets. - -[Note for North Americans: "bracket" to some English speakers, including -myself, can be round, square, curly, or pointy. Hence this usage rather than -"parentheses".] - -Non-capturing brackets use the opcode OP_BRA. Originally PCRE was limited to 99 -capturing brackets and it used a different opcode for each one. From release -3.5, the limit was removed by putting the bracket number into the data for -higher-numbered brackets. From release 7.0 all capturing brackets are handled -this way, using the single opcode OP_CBRA. - -A bracket opcode is followed by LINK_SIZE bytes which give the offset to the -next alternative OP_ALT or, if there aren't any branches, to the matching -OP_KET opcode. Each OP_ALT is followed by LINK_SIZE bytes giving the offset to -the next one, or to the OP_KET opcode. For capturing brackets, the bracket -number is a count that immediately follows the offset. - -OP_KET is used for subpatterns that do not repeat indefinitely, and OP_KETRMIN -and OP_KETRMAX are used for indefinite repetitions, minimally or maximally -respectively (see below for possessive repetitions). All three are followed by -LINK_SIZE bytes giving (as a positive number) the offset back to the matching -bracket opcode. - -If a subpattern is quantified such that it is permitted to match zero times, it -is preceded by one of OP_BRAZERO, OP_BRAMINZERO, or OP_SKIPZERO. These are -single-unit opcodes that tell the matcher that skipping the following -subpattern entirely is a valid branch. In the case of the first two, not -skipping the pattern is also valid (greedy and non-greedy). The third is used -when a pattern has the quantifier {0,0}. It cannot be entirely discarded, -because it may be called as a subroutine from elsewhere in the regex. - -A subpattern with an indefinite maximum repetition is replicated in the -compiled data its minimum number of times (or once with OP_BRAZERO if the -minimum is zero), with the final copy terminating with OP_KETRMIN or OP_KETRMAX -as appropriate. - -A subpattern with a bounded maximum repetition is replicated in a nested -fashion up to the maximum number of times, with OP_BRAZERO or OP_BRAMINZERO -before each replication after the minimum, so that, for example, (abc){2,5} is -compiled as (abc)(abc)((abc)((abc)(abc)?)?)?, except that each bracketed group -has the same number. - -When a repeated subpattern has an unbounded upper limit, it is checked to see -whether it could match an empty string. If this is the case, the opcode in the -final replication is changed to OP_SBRA or OP_SCBRA. This tells the matcher -that it needs to check for matching an empty string when it hits OP_KETRMIN or -OP_KETRMAX, and if so, to break the loop. - - -Possessive brackets -------------------- - -When a repeated group (capturing or non-capturing) is marked as possessive by -the "+" notation, e.g. (abc)++, different opcodes are used. Their names all -have POS on the end, e.g. OP_BRAPOS instead of OP_BRA and OP_SCPBRPOS instead -of OP_SCBRA. The end of such a group is marked by OP_KETRPOS. If the minimum -repetition is zero, the group is preceded by OP_BRAPOSZERO. - - -Once-only (atomic) groups -------------------------- - -These are just like other subpatterns, but they start with the opcode -OP_ONCE or OP_ONCE_NC. The former is used when there are no capturing brackets -within the atomic group; the latter when there are. The distinction is needed -for when there is a backtrack to before the group - any captures within the -group must be reset, so it is necessary to retain backtracking points inside -the group even after it is complete in order to do this. When there are no -captures in an atomic group, all the backtracking can be discarded when it is -complete. This is more efficient, and also uses less stack. - -The check for matching an empty string in an unbounded repeat is handled -entirely at runtime, so there are just these two opcodes for atomic groups. - - -Assertions ----------- - -Forward assertions are also just like other subpatterns, but starting with one -of the opcodes OP_ASSERT or OP_ASSERT_NOT. Backward assertions use the opcodes -OP_ASSERTBACK and OP_ASSERTBACK_NOT, and the first opcode inside the assertion -is OP_REVERSE, followed by a count of the number of characters to move back the -pointer in the subject string. In ASCII mode, the count is a number of units, -but in UTF-8/16 mode each character may occupy more than one unit; in UTF-32 -mode each character occupies exactly one unit. A separate count is present in -each alternative of a lookbehind assertion, allowing them to have different -fixed lengths. - - -Conditional subpatterns ------------------------ - -These are like other subpatterns, but they start with the opcode OP_COND, or -OP_SCOND for one that might match an empty string in an unbounded repeat. If -the condition is a back reference, this is stored at the start of the -subpattern using the opcode OP_CREF followed by a count containing the -reference number, provided that the reference is to a unique capturing group. -If the reference was by name and there is more than one group with that name, -OP_DNCREF is used instead. It is followed by two counts: the index in the group -names table, and the number of groups with the same name. - -If the condition is "in recursion" (coded as "(?(R)"), or "in recursion of -group x" (coded as "(?(Rx)"), the group number is stored at the start of the -subpattern using the opcode OP_RREF (with a value of zero for "the whole -pattern") or OP_DNRREF (with data as for OP_DNCREF). For a DEFINE condition, -just the single unit OP_DEF is used (it has no associated data). Otherwise, a -conditional subpattern always starts with one of the assertions. - - -Recursion ---------- - -Recursion either matches the current regex, or some subexpression. The opcode -OP_RECURSE is followed by aLINK_SIZE value that is the offset to the starting -bracket from the start of the whole pattern. From release 6.5, OP_RECURSE is -automatically wrapped inside OP_ONCE brackets, because otherwise some patterns -broke it. OP_RECURSE is also used for "subroutine" calls, even though they are -not strictly a recursion. - - -Callout -------- - -OP_CALLOUT is followed by one unit of data that holds a callout number in the -range 0 to 254 for manual callouts, or 255 for an automatic callout. In both -cases there follows a count giving the offset in the pattern string to the -start of the following item, and another count giving the length of this item. -These values make is possible for pcretest to output useful tracing information -using automatic callouts. - -Philip Hazel -November 2013 diff --git a/plugins/Pcre16/docs/LICENCE b/plugins/Pcre16/docs/LICENCE deleted file mode 100644 index 602e4ae680..0000000000 --- a/plugins/Pcre16/docs/LICENCE +++ /dev/null @@ -1,92 +0,0 @@ -PCRE LICENCE ------------- - -PCRE is a library of functions to support regular expressions whose syntax -and semantics are as close as possible to those of the Perl 5 language. - -Release 8 of PCRE is distributed under the terms of the "BSD" licence, as -specified below. The documentation for PCRE, supplied in the "doc" -directory, is distributed under the same terms as the software itself. - -The basic library functions are written in C and are freestanding. Also -included in the distribution is a set of C++ wrapper functions, and a -just-in-time compiler that can be used to optimize pattern matching. These -are both optional features that can be omitted when the library is built. - - -THE BASIC LIBRARY FUNCTIONS ---------------------------- - -Written by: Philip Hazel -Email local part: ph10 -Email domain: cam.ac.uk - -University of Cambridge Computing Service, -Cambridge, England. - -Copyright (c) 1997-2014 University of Cambridge -All rights reserved. - - -PCRE JUST-IN-TIME COMPILATION SUPPORT -------------------------------------- - -Written by: Zoltan Herczeg -Email local part: hzmester -Emain domain: freemail.hu - -Copyright(c) 2010-2014 Zoltan Herczeg -All rights reserved. - - -STACK-LESS JUST-IN-TIME COMPILER --------------------------------- - -Written by: Zoltan Herczeg -Email local part: hzmester -Emain domain: freemail.hu - -Copyright(c) 2009-2014 Zoltan Herczeg -All rights reserved. - - -THE C++ WRAPPER FUNCTIONS -------------------------- - -Contributed by: Google Inc. - -Copyright (c) 2007-2012, Google Inc. -All rights reserved. - - -THE "BSD" LICENCE ------------------ - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are met: - - * Redistributions of source code must retain the above copyright notice, - this list of conditions and the following disclaimer. - - * Redistributions in binary form must reproduce the above copyright - notice, this list of conditions and the following disclaimer in the - documentation and/or other materials provided with the distribution. - - * Neither the name of the University of Cambridge nor the name of Google - Inc. nor the names of their contributors may be used to endorse or - promote products derived from this software without specific prior - written permission. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE -LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR -CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF -SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS -INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN -CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) -ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE -POSSIBILITY OF SUCH DAMAGE. - -End diff --git a/plugins/Pcre16/docs/NEWS b/plugins/Pcre16/docs/NEWS deleted file mode 100644 index 6331e9908d..0000000000 --- a/plugins/Pcre16/docs/NEWS +++ /dev/null @@ -1,693 +0,0 @@ -News about PCRE releases ------------------------- - -Release 8.35 04-April-2014 --------------------------- - -There have been performance improvements for classes containing non-ASCII -characters and the "auto-possessification" feature has been extended. Other -minor improvements have been implemented and bugs fixed. There is a new callout -feature to enable applications to do detailed stack checks at compile time, to -avoid running out of stack for deeply nested parentheses. The JIT compiler has -been extended with experimental support for ARM-64, MIPS-64, and PPC-LE. - - -Release 8.34 15-December-2013 ------------------------------ - -As well as fixing the inevitable bugs, performance has been improved by -refactoring and extending the amount of "auto-possessification" that PCRE does. -Other notable changes: - -. Implemented PCRE_INFO_MATCH_EMPTY, which yields 1 if the pattern can match - an empty string. If it can, pcretest shows this in its information output. - -. A back reference to a named subpattern when there is more than one of the - same name now checks them in the order in which they appear in the pattern. - The first one that is set is used for the reference. Previously only the - first one was inspected. This change makes PCRE more compatible with Perl. - -. Unicode character properties were updated from Unicode 6.3.0. - -. The character VT has been added to the set of characters that match \s and - are generally treated as white space, following this same change in Perl - 5.18. There is now no difference between "Perl space" and "POSIX space". - -. Perl has changed its handling of \8 and \9. If there is no previously - encountered capturing group of those numbers, they are treated as the - literal characters 8 and 9 instead of a binary zero followed by the - literals. PCRE now does the same. - -. Following Perl, added \o{} to specify codepoints in octal, making it - possible to specify values greater than 0777 and also making them - unambiguous. - -. In UCP mode, \s was not matching two of the characters that Perl matches, - namely NEL (U+0085) and MONGOLIAN VOWEL SEPARATOR (U+180E), though they - were matched by \h. - -. Add JIT support for the 64 bit TileGX architecture. - -. Upgraded the handling of the POSIX classes [:graph:], [:print:], and - [:punct:] when PCRE_UCP is set so as to include the same characters as Perl - does in Unicode mode. - -. Perl no longer allows group names to start with digits, so I have made this - change also in PCRE. - -. Added support for [[:<:]] and [[:>:]] as used in the BSD POSIX library to - mean "start of word" and "end of word", respectively, as a transition aid. - - -Release 8.33 28-May-2013 --------------------------- - -A number of bugs are fixed, and some performance improvements have been made. -There are also some new features, of which these are the most important: - -. The behaviour of the backtracking verbs has been rationalized and - documented in more detail. - -. JIT now supports callouts and all of the backtracking verbs. - -. Unicode validation has been updated in the light of Unicode Corrigendum #9, - which points out that "non characters" are not "characters that may not - appear in Unicode strings" but rather "characters that are reserved for - internal use and have only local meaning". - -. (*LIMIT_MATCH=d) and (*LIMIT_RECURSION=d) have been added so that the - creator of a pattern can specify lower (but not higher) limits for the - matching process. - -. The PCRE_NEVER_UTF option is available to prevent pattern-writers from using - the (*UTF) feature, as this could be a security issue. - - -Release 8.32 30-November-2012 ------------------------------ - -This release fixes a number of bugs, but also has some new features. These are -the highlights: - -. There is now support for 32-bit character strings and UTF-32. Like the - 16-bit support, this is done by compiling a separate 32-bit library. - -. \X now matches a Unicode extended grapheme cluster. - -. Case-independent matching of Unicode characters that have more than one - "other case" now makes all three (or more) characters equivalent. This - applies, for example, to Greek Sigma, which has two lowercase versions. - -. Unicode character properties are updated to Unicode 6.2.0. - -. The EBCDIC support, which had decayed, has had a spring clean. - -. A number of JIT optimizations have been added, which give faster JIT - execution speed. In addition, a new direct interface to JIT execution is - available. This bypasses some of the sanity checks of pcre_exec() to give a - noticeable speed-up. - -. A number of issues in pcregrep have been fixed, making it more compatible - with GNU grep. In particular, --exclude and --include (and variants) apply - to all files now, not just those obtained from scanning a directory - recursively. In Windows environments, the default action for directories is - now "skip" instead of "read" (which provokes an error). - -. If the --only-matching (-o) option in pcregrep is specified multiple - times, each one causes appropriate output. For example, -o1 -o2 outputs the - substrings matched by the 1st and 2nd capturing parentheses. A separating - string can be specified by --om-separator (default empty). - -. When PCRE is built via Autotools using a version of gcc that has the - "visibility" feature, it is used to hide internal library functions that are - not part of the public API. - - -Release 8.31 06-July-2012 -------------------------- - -This is mainly a bug-fixing release, with a small number of developments: - -. The JIT compiler now supports partial matching and the (*MARK) and - (*COMMIT) verbs. - -. PCRE_INFO_MAXLOOKBEHIND can be used to find the longest lookbehind in a - pattern. - -. There should be a performance improvement when using the heap instead of the - stack for recursion. - -. pcregrep can now be linked with libedit as an alternative to libreadline. - -. pcregrep now has a --file-list option where the list of files to scan is - given as a file. - -. pcregrep now recognizes binary files and there are related options. - -. The Unicode tables have been updated to 6.1.0. - -As always, the full list of changes is in the ChangeLog file. - - -Release 8.30 04-February-2012 ------------------------------ - -Release 8.30 introduces a major new feature: support for 16-bit character -strings, compiled as a separate library. There are a few changes to the -8-bit library, in addition to some bug fixes. - -. The pcre_info() function, which has been obsolete for over 10 years, has - been removed. - -. When a compiled pattern was saved to a file and later reloaded on a host - with different endianness, PCRE used automatically to swap the bytes in some - of the data fields. With the advent of the 16-bit library, where more of this - swapping is needed, it is no longer done automatically. Instead, the bad - endianness is detected and a specific error is given. The user can then call - a new function called pcre_pattern_to_host_byte_order() (or an equivalent - 16-bit function) to do the swap. - -. In UTF-8 mode, the values 0xd800 to 0xdfff are not legal Unicode - code points and are now faulted. (They are the so-called "surrogates" - that are reserved for coding high values in UTF-16.) - - -Release 8.21 12-Dec-2011 ------------------------- - -This is almost entirely a bug-fix release. The only new feature is the ability -to obtain the size of the memory used by the JIT compiler. - - -Release 8.20 21-Oct-2011 ------------------------- - -The main change in this release is the inclusion of Zoltan Herczeg's -just-in-time compiler support, which can be accessed by building PCRE with ---enable-jit. Large performance benefits can be had in many situations. 8.20 -also fixes an unfortunate bug that was introduced in 8.13 as well as tidying up -a number of infelicities and differences from Perl. - - -Release 8.13 16-Aug-2011 ------------------------- - -This is mainly a bug-fix release. There has been a lot of internal refactoring. -The Unicode tables have been updated. The only new feature in the library is -the passing of *MARK information to callouts. Some additions have been made to -pcretest to make testing easier and more comprehensive. There is a new option -for pcregrep to adjust its internal buffer size. - - -Release 8.12 15-Jan-2011 ------------------------- - -This release fixes some bugs in pcregrep, one of which caused the tests to fail -on 64-bit big-endian systems. There are no changes to the code of the library. - - -Release 8.11 10-Dec-2010 ------------------------- - -A number of bugs in the library and in pcregrep have been fixed. As always, see -ChangeLog for details. The following are the non-bug-fix changes: - -. Added --match-limit and --recursion-limit to pcregrep. - -. Added an optional parentheses number to the -o and --only-matching options - of pcregrep. - -. Changed the way PCRE_PARTIAL_HARD affects the matching of $, \z, \Z, \b, and - \B. - -. Added PCRE_ERROR_SHORTUTF8 to make it possible to distinguish between a - bad UTF-8 sequence and one that is incomplete when using PCRE_PARTIAL_HARD. - -. Recognize (*NO_START_OPT) at the start of a pattern to set the PCRE_NO_ - START_OPTIMIZE option, which is now allowed at compile time - - -Release 8.10 25-Jun-2010 ------------------------- - -There are two major additions: support for (*MARK) and friends, and the option -PCRE_UCP, which changes the behaviour of \b, \d, \s, and \w (and their -opposites) so that they make use of Unicode properties. There are also a number -of lesser new features, and several bugs have been fixed. A new option, ---line-buffered, has been added to pcregrep, for use when it is connected to -pipes. - - -Release 8.02 19-Mar-2010 ------------------------- - -Another bug-fix release. - - -Release 8.01 19-Jan-2010 ------------------------- - -This is a bug-fix release. Several bugs in the code itself and some bugs and -infelicities in the build system have been fixed. - - -Release 8.00 19-Oct-09 ----------------------- - -Bugs have been fixed in the library and in pcregrep. There are also some -enhancements. Restrictions on patterns used for partial matching have been -removed, extra information is given for partial matches, the partial matching -process has been improved, and an option to make a partial match override a -full match is available. The "study" process has been enhanced by finding a -lower bound matching length. Groups with duplicate numbers may now have -duplicated names without the use of PCRE_DUPNAMES. However, they may not have -different names. The documentation has been revised to reflect these changes. -The version number has been expanded to 3 digits as it is clear that the rate -of change is not slowing down. - - -Release 7.9 11-Apr-09 ---------------------- - -Mostly bugfixes and tidies with just a couple of minor functional additions. - - -Release 7.8 05-Sep-08 ---------------------- - -More bug fixes, plus a performance improvement in Unicode character property -lookup. - - -Release 7.7 07-May-08 ---------------------- - -This is once again mainly a bug-fix release, but there are a couple of new -features. - - -Release 7.6 28-Jan-08 ---------------------- - -The main reason for having this release so soon after 7.5 is because it fixes a -potential buffer overflow problem in pcre_compile() when run in UTF-8 mode. In -addition, the CMake configuration files have been brought up to date. - - -Release 7.5 10-Jan-08 ---------------------- - -This is mainly a bug-fix release. However the ability to link pcregrep with -libz or libbz2 and the ability to link pcretest with libreadline have been -added. Also the --line-offsets and --file-offsets options were added to -pcregrep. - - -Release 7.4 21-Sep-07 ---------------------- - -The only change of specification is the addition of options to control whether -\R matches any Unicode line ending (the default) or just CR, LF, and CRLF. -Otherwise, the changes are bug fixes and a refactoring to reduce the number of -relocations needed in a shared library. There have also been some documentation -updates, in particular, some more information about using CMake to build PCRE -has been added to the NON-UNIX-USE file. - - -Release 7.3 28-Aug-07 ---------------------- - -Most changes are bug fixes. Some that are not: - -1. There is some support for Perl 5.10's experimental "backtracking control - verbs" such as (*PRUNE). - -2. UTF-8 checking is now as per RFC 3629 instead of RFC 2279; this is more - restrictive in the strings it accepts. - -3. Checking for potential integer overflow has been made more dynamic, and as a - consequence there is no longer a hard limit on the size of a subpattern that - has a limited repeat count. - -4. When CRLF is a valid line-ending sequence, pcre_exec() and pcre_dfa_exec() - no longer advance by two characters instead of one when an unanchored match - fails at CRLF if there are explicit CR or LF matches within the pattern. - This gets rid of some anomalous effects that previously occurred. - -5. Some PCRE-specific settings for varying the newline options at the start of - a pattern have been added. - - -Release 7.2 19-Jun-07 ---------------------- - -WARNING: saved patterns that were compiled by earlier versions of PCRE must be -recompiled for use with 7.2 (necessitated by the addition of \K, \h, \H, \v, -and \V). - -Correction to the notes for 7.1: the note about shared libraries for Windows is -wrong. Previously, three libraries were built, but each could function -independently. For example, the pcreposix library also included all the -functions from the basic pcre library. The change is that the three libraries -are no longer independent. They are like the Unix libraries. To use the -pcreposix functions, for example, you need to link with both the pcreposix and -the basic pcre library. - -Some more features from Perl 5.10 have been added: - - (?-n) and (?+n) relative references for recursion and subroutines. - - (?(-n) and (?(+n) relative references as conditions. - - \k{name} and \g{name} are synonyms for \k<name>. - - \K to reset the start of the matched string; for example, (foo)\Kbar - matches bar preceded by foo, but only sets bar as the matched string. - - (?| introduces a group where the capturing parentheses in each alternative - start from the same number; for example, (?|(abc)|(xyz)) sets capturing - parentheses number 1 in both cases. - - \h, \H, \v, \V match horizontal and vertical whitespace, respectively. - - -Release 7.1 24-Apr-07 ---------------------- - -There is only one new feature in this release: a linebreak setting of -PCRE_NEWLINE_ANYCRLF. It is a cut-down version of PCRE_NEWLINE_ANY, which -recognizes only CRLF, CR, and LF as linebreaks. - -A few bugs are fixed (see ChangeLog for details), but the major change is a -complete re-implementation of the build system. This now has full Autotools -support and so is now "standard" in some sense. It should help with compiling -PCRE in a wide variety of environments. - -NOTE: when building shared libraries for Windows, three dlls are now built, -called libpcre, libpcreposix, and libpcrecpp. Previously, everything was -included in a single dll. - -Another important change is that the dftables auxiliary program is no longer -compiled and run at "make" time by default. Instead, a default set of character -tables (assuming ASCII coding) is used. If you want to use dftables to generate -the character tables as previously, add --enable-rebuild-chartables to the -"configure" command. You must do this if you are compiling PCRE to run on a -system that uses EBCDIC code. - -There is a discussion about character tables in the README file. The default is -not to use dftables so that that there is no problem when cross-compiling. - - -Release 7.0 19-Dec-06 ---------------------- - -This release has a new major number because there have been some internal -upheavals to facilitate the addition of new optimizations and other facilities, -and to make subsequent maintenance and extension easier. Compilation is likely -to be a bit slower, but there should be no major effect on runtime performance. -Previously compiled patterns are NOT upwards compatible with this release. If -you have saved compiled patterns from a previous release, you will have to -re-compile them. Important changes that are visible to users are: - -1. The Unicode property tables have been updated to Unicode 5.0.0, which adds - some more scripts. - -2. The option PCRE_NEWLINE_ANY causes PCRE to recognize any Unicode newline - sequence as a newline. - -3. The \R escape matches a single Unicode newline sequence as a single unit. - -4. New features that will appear in Perl 5.10 are now in PCRE. These include - alternative Perl syntax for named parentheses, and Perl syntax for - recursion. - -5. The C++ wrapper interface has been extended by the addition of a - QuoteMeta function and the ability to allow copy construction and - assignment. - -For a complete list of changes, see the ChangeLog file. - - -Release 6.7 04-Jul-06 ---------------------- - -The main additions to this release are the ability to use the same name for -multiple sets of parentheses, and support for CRLF line endings in both the -library and pcregrep (and in pcretest for testing). - -Thanks to Ian Taylor, the stack usage for many kinds of pattern has been -significantly reduced for certain subject strings. - - -Release 6.5 01-Feb-06 ---------------------- - -Important changes in this release: - -1. A number of new features have been added to pcregrep. - -2. The Unicode property tables have been updated to Unicode 4.1.0, and the - supported properties have been extended with script names such as "Arabic", - and the derived properties "Any" and "L&". This has necessitated a change to - the interal format of compiled patterns. Any saved compiled patterns that - use \p or \P must be recompiled. - -3. The specification of recursion in patterns has been changed so that all - recursive subpatterns are automatically treated as atomic groups. Thus, for - example, (?R) is treated as if it were (?>(?R)). This is necessary because - otherwise there are situations where recursion does not work. - -See the ChangeLog for a complete list of changes, which include a number of bug -fixes and tidies. - - -Release 6.0 07-Jun-05 ---------------------- - -The release number has been increased to 6.0 because of the addition of several -major new pieces of functionality. - -A new function, pcre_dfa_exec(), which implements pattern matching using a DFA -algorithm, has been added. This has a number of advantages for certain cases, -though it does run more slowly, and lacks the ability to capture substrings. On -the other hand, it does find all matches, not just the first, and it works -better for partial matching. The pcrematching man page discusses the -differences. - -The pcretest program has been enhanced so that it can make use of the new -pcre_dfa_exec() matching function and the extra features it provides. - -The distribution now includes a C++ wrapper library. This is built -automatically if a C++ compiler is found. The pcrecpp man page discusses this -interface. - -The code itself has been re-organized into many more files, one for each -function, so it no longer requires everything to be linked in when static -linkage is used. As a consequence, some internal functions have had to have -their names exposed. These functions all have names starting with _pcre_. They -are undocumented, and are not intended for use by outside callers. - -The pcregrep program has been enhanced with new functionality such as -multiline-matching and options for output more matching context. See the -ChangeLog for a complete list of changes to the library and the utility -programs. - - -Release 5.0 13-Sep-04 ---------------------- - -The licence under which PCRE is released has been changed to the more -conventional "BSD" licence. - -In the code, some bugs have been fixed, and there are also some major changes -in this release (which is why I've increased the number to 5.0). Some changes -are internal rearrangements, and some provide a number of new facilities. The -new features are: - -1. There's an "automatic callout" feature that inserts callouts before every - item in the regex, and there's a new callout field that gives the position - in the pattern - useful for debugging and tracing. - -2. The extra_data structure can now be used to pass in a set of character - tables at exec time. This is useful if compiled regex are saved and re-used - at a later time when the tables may not be at the same address. If the - default internal tables are used, the pointer saved with the compiled - pattern is now set to NULL, which means that you don't need to do anything - special unless you are using custom tables. - -3. It is possible, with some restrictions on the content of the regex, to - request "partial" matching. A special return code is given if all of the - subject string matched part of the regex. This could be useful for testing - an input field as it is being typed. - -4. There is now some optional support for Unicode character properties, which - means that the patterns items such as \p{Lu} and \X can now be used. Only - the general category properties are supported. If PCRE is compiled with this - support, an additional 90K data structure is include, which increases the - size of the library dramatically. - -5. There is support for saving compiled patterns and re-using them later. - -6. There is support for running regular expressions that were compiled on a - different host with the opposite endianness. - -7. The pcretest program has been extended to accommodate the new features. - -The main internal rearrangement is that sequences of literal characters are no -longer handled as strings. Instead, each character is handled on its own. This -makes some UTF-8 handling easier, and makes the support of partial matching -possible. Compiled patterns containing long literal strings will be larger as a -result of this change; I hope that performance will not be much affected. - - -Release 4.5 01-Dec-03 ---------------------- - -Again mainly a bug-fix and tidying release, with only a couple of new features: - -1. It's possible now to compile PCRE so that it does not use recursive -function calls when matching. Instead it gets memory from the heap. This slows -things down, but may be necessary on systems with limited stacks. - -2. UTF-8 string checking has been tightened to reject overlong sequences and to -check that a starting offset points to the start of a character. Failure of the -latter returns a new error code: PCRE_ERROR_BADUTF8_OFFSET. - -3. PCRE can now be compiled for systems that use EBCDIC code. - - -Release 4.4 21-Aug-03 ---------------------- - -This is mainly a bug-fix and tidying release. The only new feature is that PCRE -checks UTF-8 strings for validity by default. There is an option to suppress -this, just in case anybody wants that teeny extra bit of performance. - - -Releases 4.1 - 4.3 ------------------- - -Sorry, I forgot about updating the NEWS file for these releases. Please take a -look at ChangeLog. - - -Release 4.0 17-Feb-03 ---------------------- - -There have been a lot of changes for the 4.0 release, adding additional -functionality and mending bugs. Below is a list of the highlights of the new -functionality. For full details of these features, please consult the -documentation. For a complete list of changes, see the ChangeLog file. - -1. Support for Perl's \Q...\E escapes. - -2. "Possessive quantifiers" ?+, *+, ++, and {,}+ which come from Sun's Java -package. They provide some syntactic sugar for simple cases of "atomic -grouping". - -3. Support for the \G assertion. It is true when the current matching position -is at the start point of the match. - -4. A new feature that provides some of the functionality that Perl provides -with (?{...}). The facility is termed a "callout". The way it is done in PCRE -is for the caller to provide an optional function, by setting pcre_callout to -its entry point. To get the function called, the regex must include (?C) at -appropriate points. - -5. Support for recursive calls to individual subpatterns. This makes it really -easy to get totally confused. - -6. Support for named subpatterns. The Python syntax (?P<name>...) is used to -name a group. - -7. Several extensions to UTF-8 support; it is now fairly complete. There is an -option for pcregrep to make it operate in UTF-8 mode. - -8. The single man page has been split into a number of separate man pages. -These also give rise to individual HTML pages which are put in a separate -directory. There is an index.html page that lists them all. Some hyperlinking -between the pages has been installed. - - -Release 3.5 15-Aug-01 ---------------------- - -1. The configuring system has been upgraded to use later versions of autoconf -and libtool. By default it builds both a shared and a static library if the OS -supports it. You can use --disable-shared or --disable-static on the configure -command if you want only one of them. - -2. The pcretest utility is now installed along with pcregrep because it is -useful for users (to test regexs) and by doing this, it automatically gets -relinked by libtool. The documentation has been turned into a man page, so -there are now .1, .txt, and .html versions in /doc. - -3. Upgrades to pcregrep: - (i) Added long-form option names like gnu grep. - (ii) Added --help to list all options with an explanatory phrase. - (iii) Added -r, --recursive to recurse into sub-directories. - (iv) Added -f, --file to read patterns from a file. - -4. Added --enable-newline-is-cr and --enable-newline-is-lf to the configure -script, to force use of CR or LF instead of \n in the source. On non-Unix -systems, the value can be set in config.h. - -5. The limit of 200 on non-capturing parentheses is a _nesting_ limit, not an -absolute limit. Changed the text of the error message to make this clear, and -likewise updated the man page. - -6. The limit of 99 on the number of capturing subpatterns has been removed. -The new limit is 65535, which I hope will not be a "real" limit. - - -Release 3.3 01-Aug-00 ---------------------- - -There is some support for UTF-8 character strings. This is incomplete and -experimental. The documentation describes what is and what is not implemented. -Otherwise, this is just a bug-fixing release. - - -Release 3.0 01-Feb-00 ---------------------- - -1. A "configure" script is now used to configure PCRE for Unix systems. It -builds a Makefile, a config.h file, and the pcre-config script. - -2. PCRE is built as a shared library by default. - -3. There is support for POSIX classes such as [:alpha:]. - -5. There is an experimental recursion feature. - ----------------------------------------------------------------------------- - IMPORTANT FOR THOSE UPGRADING FROM VERSIONS BEFORE 2.00 - -Please note that there has been a change in the API such that a larger -ovector is required at matching time, to provide some additional workspace. -The new man page has details. This change was necessary in order to support -some of the new functionality in Perl 5.005. - - IMPORTANT FOR THOSE UPGRADING FROM VERSION 2.00 - -Another (I hope this is the last!) change has been made to the API for the -pcre_compile() function. An additional argument has been added to make it -possible to pass over a pointer to character tables built in the current -locale by pcre_maketables(). To use the default tables, this new argument -should be passed as NULL. - - IMPORTANT FOR THOSE UPGRADING FROM VERSION 2.05 - -Yet another (and again I hope this really is the last) change has been made -to the API for the pcre_exec() function. An additional argument has been -added to make it possible to start the match other than at the start of the -subject string. This is important if there are lookbehinds. The new man -page has the details, but you just want to convert existing programs, all -you need to do is to stick in a new fifth argument to pcre_exec(), with a -value of zero. For example, change - - pcre_exec(pattern, extra, subject, length, options, ovec, ovecsize) -to - pcre_exec(pattern, extra, subject, length, 0, options, ovec, ovecsize) - -**** diff --git a/plugins/Pcre16/docs/NON-AUTOTOOLS-BUILD b/plugins/Pcre16/docs/NON-AUTOTOOLS-BUILD deleted file mode 100644 index cddf3e0699..0000000000 --- a/plugins/Pcre16/docs/NON-AUTOTOOLS-BUILD +++ /dev/null @@ -1,764 +0,0 @@ -Building PCRE without using autotools -------------------------------------- - -This document contains the following sections: - - General - Generic instructions for the PCRE C library - The C++ wrapper functions - Building for virtual Pascal - Stack size in Windows environments - Linking programs in Windows environments - Calling conventions in Windows environments - Comments about Win32 builds - Building PCRE on Windows with CMake - Use of relative paths with CMake on Windows - Testing with RunTest.bat - Building under Windows CE with Visual Studio 200x - Building under Windows with BCC5.5 - Building using Borland C++ Builder 2007 (CB2007) and higher - Building PCRE on OpenVMS - Building PCRE on Stratus OpenVOS - Building PCRE on native z/OS and z/VM - - -GENERAL - -I (Philip Hazel) have no experience of Windows or VMS sytems and how their -libraries work. The items in the PCRE distribution and Makefile that relate to -anything other than Linux systems are untested by me. - -There are some other comments and files (including some documentation in CHM -format) in the Contrib directory on the FTP site: - - ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib - -The basic PCRE library consists entirely of code written in Standard C, and so -should compile successfully on any system that has a Standard C compiler and -library. The C++ wrapper functions are a separate issue (see below). - -The PCRE distribution includes a "configure" file for use by the configure/make -(autotools) build system, as found in many Unix-like environments. The README -file contains information about the options for "configure". - -There is also support for CMake, which some users prefer, especially in Windows -environments, though it can also be run in Unix-like environments. See the -section entitled "Building PCRE on Windows with CMake" below. - -Versions of config.h and pcre.h are distributed in the PCRE tarballs under the -names config.h.generic and pcre.h.generic. These are provided for those who -build PCRE without using "configure" or CMake. If you use "configure" or CMake, -the .generic versions are not used. - - -GENERIC INSTRUCTIONS FOR THE PCRE C LIBRARY - -The following are generic instructions for building the PCRE C library "by -hand". If you are going to use CMake, this section does not apply to you; you -can skip ahead to the CMake section. - - (1) Copy or rename the file config.h.generic as config.h, and edit the macro - settings that it contains to whatever is appropriate for your environment. - - In particular, you can alter the definition of the NEWLINE macro to - specify what character(s) you want to be interpreted as line terminators. - In an EBCDIC environment, you MUST change NEWLINE, because its default - value is 10, an ASCII LF. The usual EBCDIC newline character is 21 (0x15, - NL), though in some cases it may be 37 (0x25). - - When you compile any of the PCRE modules, you must specify -DHAVE_CONFIG_H - to your compiler so that config.h is included in the sources. - - An alternative approach is not to edit config.h, but to use -D on the - compiler command line to make any changes that you need to the - configuration options. In this case -DHAVE_CONFIG_H must not be set. - - NOTE: There have been occasions when the way in which certain parameters - in config.h are used has changed between releases. (In the configure/make - world, this is handled automatically.) When upgrading to a new release, - you are strongly advised to review config.h.generic before re-using what - you had previously. - - (2) Copy or rename the file pcre.h.generic as pcre.h. - - (3) EITHER: - Copy or rename file pcre_chartables.c.dist as pcre_chartables.c. - - OR: - Compile dftables.c as a stand-alone program (using -DHAVE_CONFIG_H if - you have set up config.h), and then run it with the single argument - "pcre_chartables.c". This generates a set of standard character tables - and writes them to that file. The tables are generated using the default - C locale for your system. If you want to use a locale that is specified - by LC_xxx environment variables, add the -L option to the dftables - command. You must use this method if you are building on a system that - uses EBCDIC code. - - The tables in pcre_chartables.c are defaults. The caller of PCRE can - specify alternative tables at run time. - - (4) Ensure that you have the following header files: - - pcre_internal.h - ucp.h - - (5) For an 8-bit library, compile the following source files, setting - -DHAVE_CONFIG_H as a compiler option if you have set up config.h with your - configuration, or else use other -D settings to change the configuration - as required. - - pcre_byte_order.c - pcre_chartables.c - pcre_compile.c - pcre_config.c - pcre_dfa_exec.c - pcre_exec.c - pcre_fullinfo.c - pcre_get.c - pcre_globals.c - pcre_jit_compile.c - pcre_maketables.c - pcre_newline.c - pcre_ord2utf8.c - pcre_refcount.c - pcre_string_utils.c - pcre_study.c - pcre_tables.c - pcre_ucd.c - pcre_valid_utf8.c - pcre_version.c - pcre_xclass.c - - Make sure that you include -I. in the compiler command (or equivalent for - an unusual compiler) so that all included PCRE header files are first - sought in the current directory. Otherwise you run the risk of picking up - a previously-installed file from somewhere else. - - Note that you must still compile pcre_jit_compile.c, even if you have not - defined SUPPORT_JIT in config.h, because when JIT support is not - configured, dummy functions are compiled. When JIT support IS configured, - pcre_jit_compile.c #includes sources from the sljit subdirectory, where - there should be 16 files, all of whose names begin with "sljit". - - (6) Now link all the compiled code into an object library in whichever form - your system keeps such libraries. This is the basic PCRE C 8-bit library. - If your system has static and shared libraries, you may have to do this - once for each type. - - (7) If you want to build a 16-bit library (as well as, or instead of the 8-bit - or 32-bit libraries) repeat steps 5-6 with the following files: - - pcre16_byte_order.c - pcre16_chartables.c - pcre16_compile.c - pcre16_config.c - pcre16_dfa_exec.c - pcre16_exec.c - pcre16_fullinfo.c - pcre16_get.c - pcre16_globals.c - pcre16_jit_compile.c - pcre16_maketables.c - pcre16_newline.c - pcre16_ord2utf16.c - pcre16_refcount.c - pcre16_string_utils.c - pcre16_study.c - pcre16_tables.c - pcre16_ucd.c - pcre16_utf16_utils.c - pcre16_valid_utf16.c - pcre16_version.c - pcre16_xclass.c - - (8) If you want to build a 32-bit library (as well as, or instead of the 8-bit - or 16-bit libraries) repeat steps 5-6 with the following files: - - pcre32_byte_order.c - pcre32_chartables.c - pcre32_compile.c - pcre32_config.c - pcre32_dfa_exec.c - pcre32_exec.c - pcre32_fullinfo.c - pcre32_get.c - pcre32_globals.c - pcre32_jit_compile.c - pcre32_maketables.c - pcre32_newline.c - pcre32_ord2utf32.c - pcre32_refcount.c - pcre32_string_utils.c - pcre32_study.c - pcre32_tables.c - pcre32_ucd.c - pcre32_utf32_utils.c - pcre32_valid_utf32.c - pcre32_version.c - pcre32_xclass.c - - (9) If you want to build the POSIX wrapper functions (which apply only to the - 8-bit library), ensure that you have the pcreposix.h file and then compile - pcreposix.c (remembering -DHAVE_CONFIG_H if necessary). Link the result - (on its own) as the pcreposix library. - -(10) The pcretest program can be linked with any combination of the 8-bit, - 16-bit and 32-bit libraries (depending on what you selected in config.h). - Compile pcretest.c and pcre_printint.c (again, don't forget - -DHAVE_CONFIG_H) and link them together with the appropriate library/ies. - If you compiled an 8-bit library, pcretest also needs the pcreposix - wrapper library unless you compiled it with -DNOPOSIX. - -(11) Run pcretest on the testinput files in the testdata directory, and check - that the output matches the corresponding testoutput files. There are - comments about what each test does in the section entitled "Testing PCRE" - in the README file. If you compiled more than one of the 8-bit, 16-bit and - 32-bit libraries, you need to run pcretest with the -16 option to do - 16-bit tests and with the -32 option to do 32-bit tests. - - Some tests are relevant only when certain build-time options are selected. - For example, test 4 is for UTF-8/UTF-16/UTF-32 support, and will not run - if you have built PCRE without it. See the comments at the start of each - testinput file. If you have a suitable Unix-like shell, the RunTest script - will run the appropriate tests for you. The command "RunTest list" will - output a list of all the tests. - - Note that the supplied files are in Unix format, with just LF characters - as line terminators. You may need to edit them to change this if your - system uses a different convention. If you are using Windows, you probably - should use the wintestinput3 file instead of testinput3 (and the - corresponding output file). This is a locale test; wintestinput3 sets the - locale to "french" rather than "fr_FR", and there some minor output - differences. - -(12) If you have built PCRE with SUPPORT_JIT, the JIT features will be tested - by the testdata files. However, you might also like to build and run - the freestanding JIT test program, pcre_jit_test.c. - -(13) If you want to use the pcregrep command, compile and link pcregrep.c; it - uses only the basic 8-bit PCRE library (it does not need the pcreposix - library). - - -THE C++ WRAPPER FUNCTIONS - -The PCRE distribution also contains some C++ wrapper functions and tests, -applicable to the 8-bit library, which were contributed by Google Inc. On a -system that can use "configure" and "make", the functions are automatically -built into a library called pcrecpp. It should be straightforward to compile -the .cc files manually on other systems. The files called xxx_unittest.cc are -test programs for each of the corresponding xxx.cc files. - - -BUILDING FOR VIRTUAL PASCAL - -A script for building PCRE using Borland's C++ compiler for use with VPASCAL -was contributed by Alexander Tokarev. Stefan Weber updated the script and added -additional files. The following files in the distribution are for building PCRE -for use with VP/Borland: makevp_c.txt, makevp_l.txt, makevp.bat, pcregexp.pas. - - -STACK SIZE IN WINDOWS ENVIRONMENTS - -The default processor stack size of 1Mb in some Windows environments is too -small for matching patterns that need much recursion. In particular, test 2 may -fail because of this. Normally, running out of stack causes a crash, but there -have been cases where the test program has just died silently. See your linker -documentation for how to increase stack size if you experience problems. The -Linux default of 8Mb is a reasonable choice for the stack, though even that can -be too small for some pattern/subject combinations. - -PCRE has a compile configuration option to disable the use of stack for -recursion so that heap is used instead. However, pattern matching is -significantly slower when this is done. There is more about stack usage in the -"pcrestack" documentation. - - -LINKING PROGRAMS IN WINDOWS ENVIRONMENTS - -If you want to statically link a program against a PCRE library in the form of -a non-dll .a file, you must define PCRE_STATIC before including pcre.h or -pcrecpp.h, otherwise the pcre_malloc() and pcre_free() exported functions will -be declared __declspec(dllimport), with unwanted results. - - -CALLING CONVENTIONS IN WINDOWS ENVIRONMENTS - -It is possible to compile programs to use different calling conventions using -MSVC. Search the web for "calling conventions" for more information. To make it -easier to change the calling convention for the exported functions in the -PCRE library, the macro PCRE_CALL_CONVENTION is present in all the external -definitions. It can be set externally when compiling (e.g. in CFLAGS). If it is -not set, it defaults to empty; the default calling convention is then used -(which is what is wanted most of the time). - - -COMMENTS ABOUT WIN32 BUILDS (see also "BUILDING PCRE ON WINDOWS WITH CMAKE") - -There are two ways of building PCRE using the "configure, make, make install" -paradigm on Windows systems: using MinGW or using Cygwin. These are not at all -the same thing; they are completely different from each other. There is also -support for building using CMake, which some users find a more straightforward -way of building PCRE under Windows. - -The MinGW home page (http://www.mingw.org/) says this: - - MinGW: A collection of freely available and freely distributable Windows - specific header files and import libraries combined with GNU toolsets that - allow one to produce native Windows programs that do not rely on any - 3rd-party C runtime DLLs. - -The Cygwin home page (http://www.cygwin.com/) says this: - - Cygwin is a Linux-like environment for Windows. It consists of two parts: - - . A DLL (cygwin1.dll) which acts as a Linux API emulation layer providing - substantial Linux API functionality - - . A collection of tools which provide Linux look and feel. - - The Cygwin DLL currently works with all recent, commercially released x86 32 - bit and 64 bit versions of Windows, with the exception of Windows CE. - -On both MinGW and Cygwin, PCRE should build correctly using: - - ./configure && make && make install - -This should create two libraries called libpcre and libpcreposix, and, if you -have enabled building the C++ wrapper, a third one called libpcrecpp. These are -independent libraries: when you link with libpcreposix or libpcrecpp you must -also link with libpcre, which contains the basic functions. (Some earlier -releases of PCRE included the basic libpcre functions in libpcreposix. This no -longer happens.) - -A user submitted a special-purpose patch that makes it easy to create -"pcre.dll" under mingw32 using the "msys" environment. It provides "pcre.dll" -as a special target. If you use this target, no other files are built, and in -particular, the pcretest and pcregrep programs are not built. An example of how -this might be used is: - - ./configure --enable-utf --disable-cpp CFLAGS="-03 -s"; make pcre.dll - -Using Cygwin's compiler generates libraries and executables that depend on -cygwin1.dll. If a library that is generated this way is distributed, -cygwin1.dll has to be distributed as well. Since cygwin1.dll is under the GPL -licence, this forces not only PCRE to be under the GPL, but also the entire -application. A distributor who wants to keep their own code proprietary must -purchase an appropriate Cygwin licence. - -MinGW has no such restrictions. The MinGW compiler generates a library or -executable that can run standalone on Windows without any third party dll or -licensing issues. - -But there is more complication: - -If a Cygwin user uses the -mno-cygwin Cygwin gcc flag, what that really does is -to tell Cygwin's gcc to use the MinGW gcc. Cygwin's gcc is only acting as a -front end to MinGW's gcc (if you install Cygwin's gcc, you get both Cygwin's -gcc and MinGW's gcc). So, a user can: - -. Build native binaries by using MinGW or by getting Cygwin and using - -mno-cygwin. - -. Build binaries that depend on cygwin1.dll by using Cygwin with the normal - compiler flags. - -The test files that are supplied with PCRE are in UNIX format, with LF -characters as line terminators. Unless your PCRE library uses a default newline -option that includes LF as a valid newline, it may be necessary to change the -line terminators in the test files to get some of the tests to work. - - -BUILDING PCRE ON WINDOWS WITH CMAKE - -CMake is an alternative configuration facility that can be used instead of -"configure". CMake creates project files (make files, solution files, etc.) -tailored to numerous development environments, including Visual Studio, -Borland, Msys, MinGW, NMake, and Unix. If possible, use short paths with no -spaces in the names for your CMake installation and your PCRE source and build -directories. - -The following instructions were contributed by a PCRE user. If they are not -followed exactly, errors may occur. In the event that errors do occur, it is -recommended that you delete the CMake cache before attempting to repeat the -CMake build process. In the CMake GUI, the cache can be deleted by selecting -"File > Delete Cache". - -1. Install the latest CMake version available from http://www.cmake.org/, and - ensure that cmake\bin is on your path. - -2. Unzip (retaining folder structure) the PCRE source tree into a source - directory such as C:\pcre. You should ensure your local date and time - is not earlier than the file dates in your source dir if the release is - very new. - -3. Create a new, empty build directory, preferably a subdirectory of the - source dir. For example, C:\pcre\pcre-xx\build. - -4. Run cmake-gui from the Shell envirornment of your build tool, for example, - Msys for Msys/MinGW or Visual Studio Command Prompt for VC/VC++. Do not try - to start Cmake from the Windows Start menu, as this can lead to errors. - -5. Enter C:\pcre\pcre-xx and C:\pcre\pcre-xx\build for the source and build - directories, respectively. - -6. Hit the "Configure" button. - -7. Select the particular IDE / build tool that you are using (Visual - Studio, MSYS makefiles, MinGW makefiles, etc.) - -8. The GUI will then list several configuration options. This is where - you can enable UTF-8 support or other PCRE optional features. - -9. Hit "Configure" again. The adjacent "Generate" button should now be - active. - -10. Hit "Generate". - -11. The build directory should now contain a usable build system, be it a - solution file for Visual Studio, makefiles for MinGW, etc. Exit from - cmake-gui and use the generated build system with your compiler or IDE. - E.g., for MinGW you can run "make", or for Visual Studio, open the PCRE - solution, select the desired configuration (Debug, or Release, etc.) and - build the ALL_BUILD project. - -12. If during configuration with cmake-gui you've elected to build the test - programs, you can execute them by building the test project. E.g., for - MinGW: "make test"; for Visual Studio build the RUN_TESTS project. The - most recent build configuration is targeted by the tests. A summary of - test results is presented. Complete test output is subsequently - available for review in Testing\Temporary under your build dir. - - -USE OF RELATIVE PATHS WITH CMAKE ON WINDOWS - -A PCRE user comments as follows: I thought that others may want to know the -current state of CMAKE_USE_RELATIVE_PATHS support on Windows. Here it is: - --- AdditionalIncludeDirectories is only partially modified (only the - first path - see below) --- Only some of the contained file paths are modified - shown below for - pcre.vcproj --- It properly modifies - -I am sure CMake people can fix that if they want to. Until then one will -need to replace existing absolute paths in project files with relative -paths manually (e.g. from VS) - relative to project file location. I did -just that before being told to try CMAKE_USE_RELATIVE_PATHS. Not a big -deal. - -AdditionalIncludeDirectories="E:\builds\pcre\build;E:\builds\pcre\pcre-7.5;" -AdditionalIncludeDirectories=".;E:\builds\pcre\pcre-7.5;" - -RelativePath="pcre.h" -RelativePath="pcre_chartables.c" -RelativePath="pcre_chartables.c.rule" - - -TESTING WITH RUNTEST.BAT - -If configured with CMake, building the test project ("make test" or building -ALL_TESTS in Visual Studio) creates (and runs) pcre_test.bat (and depending -on your configuration options, possibly other test programs) in the build -directory. Pcre_test.bat runs RunTest.Bat with correct source and exe paths. - -For manual testing with RunTest.bat, provided the build dir is a subdirectory -of the source directory: Open command shell window. Chdir to the location -of your pcretest.exe and pcregrep.exe programs. Call RunTest.bat with -"..\RunTest.Bat" or "..\..\RunTest.bat" as appropriate. - -To run only a particular test with RunTest.Bat provide a test number argument. - -Otherwise: - -1. Copy RunTest.bat into the directory where pcretest.exe and pcregrep.exe - have been created. - -2. Edit RunTest.bat to indentify the full or relative location of - the pcre source (wherein which the testdata folder resides), e.g.: - - set srcdir=C:\pcre\pcre-8.20 - -3. In a Windows command environment, chdir to the location of your bat and - exe programs. - -4. Run RunTest.bat. Test outputs will automatically be compared to expected - results, and discrepancies will be identified in the console output. - -To independently test the just-in-time compiler, run pcre_jit_test.exe. -To test pcrecpp, run pcrecpp_unittest.exe, pcre_stringpiece_unittest.exe and -pcre_scanner_unittest.exe. - - -BUILDING UNDER WINDOWS CE WITH VISUAL STUDIO 200x - -Vincent Richomme sent a zip archive of files to help with this process. They -can be found in the file "pcre-vsbuild.zip" in the Contrib directory of the FTP -site. - - -BUILDING UNDER WINDOWS WITH BCC5.5 - -Michael Roy sent these comments about building PCRE under Windows with BCC5.5: - -Some of the core BCC libraries have a version of PCRE from 1998 built in, which -can lead to pcre_exec() giving an erroneous PCRE_ERROR_NULL from a version -mismatch. I'm including an easy workaround below, if you'd like to include it -in the non-unix instructions: - -When linking a project with BCC5.5, pcre.lib must be included before any of the -libraries cw32.lib, cw32i.lib, cw32mt.lib, and cw32mti.lib on the command line. - - -BUILDING USING BORLAND C++ BUILDER 2007 (CB2007) AND HIGHER - -A PCRE user sent these comments about this environment (see also the comment -from another user that follows them): - -The XE versions of C++ Builder come with a RegularExpressionsCore class which -contain a version of TPerlRegEx. However, direct use of the C PCRE library may -be desirable. - -The default makevp.bat, however, supplied with PCRE builds a version of PCRE -that is not usable with any version of C++ Builder because the compiler ships -with an embedded version of PCRE, version 2.01 from 1998! [See also the note -about BCC5.5 above.] If you want to use PCRE you'll need to rename the -functions (pcre_compile to pcre_compile_bcc, etc) or do as I have done and just -use the 16 bit versions. I'm using std::wstring everywhere anyway. Since the -embedded version of PCRE does not have the 16 bit function names, there is no -conflict. - -Building PCRE using a C++ Builder static library project file (recommended): - -1. Rename or remove pcre.h, pcreposi.h, and pcreposix.h from your C++ Builder -original include path. - -2. Download PCRE from pcre.org and extract to a directory. - -3. Rename pcre_chartables.c.dist to pcre_chartables.c, pcre.h.generic to -pcre.h, and config.h.generic to config.h. - -4. Edit pcre.h and pcre_config.c so that they include config.h. - -5. Edit config.h like so: - -Comment out the following lines: -#define PACKAGE "pcre" -#define PACKAGE_BUGREPORT "" -#define PACKAGE_NAME "PCRE" -#define PACKAGE_STRING "PCRE 8.32" -#define PACKAGE_TARNAME "pcre" -#define PACKAGE_URL "" -#define PACKAGE_VERSION "8.32" - -Add the following lines: -#ifndef SUPPORT_UTF -#define SUPPORT_UTF 100 // any value is fine -#endif - -#ifndef SUPPORT_UCP -#define SUPPORT_UCP 101 // any value is fine -#endif - -#ifndef SUPPORT_UCP -#define SUPPORT_PCRE16 102 // any value is fine -#endif - -#ifndef SUPPORT_UTF8 -#define SUPPORT_UTF8 103 // any value is fine -#endif - -6. Build a C++ Builder project using the IDE. Go to File / New / Other and -choose Static Library. You can name it pcre.cbproj or whatever. Now set your -paths by going to Project / Options. Set the Include path. Do this from the -"Base" option to apply to both Release and Debug builds. Now add the following -files to the project: - -pcre.h -pcre16_byte_order.c -pcre16_chartables.c -pcre16_compile.c -pcre16_config.c -pcre16_dfa_exec.c -pcre16_exec.c -pcre16_fullinfo.c -pcre16_get.c -pcre16_globals.c -pcre16_maketables.c -pcre16_newline.c -pcre16_ord2utf16.c -pcre16_printint.c -pcre16_refcount.c -pcre16_string_utils.c -pcre16_study.c -pcre16_tables.c -pcre16_ucd.c -pcre16_utf16_utils.c -pcre16_valid_utf16.c -pcre16_version.c -pcre16_xclass.c - -//Optional -pcre_version.c - -7. After compiling the .lib file, copy the .lib and header files to a project -you want to use PCRE with. Enjoy. - -Optional ... Building PCRE using the makevp.bat file: - -1. Edit makevp_c.txt and makevp_l.txt and change all the names to the 16 bit -versions. - -2. Edit makevp.bat and set the path to C++ Builder. Run makevp.bat. - -Another PCRE user added this comment: - -Another approach I successfully used for some years with BCB 5 and 6 was to -make sure that include and library paths of PCRE are configured before the -default paths of the IDE in the dialogs where one can manage those paths. -Afterwards one can open the project files using a text editor and manually add -the self created library for pcre itself, pcrecpp doesn't ship with the IDE, in -the library nodes where the IDE manages its own libraries to link against in -front of the IDE-own libraries. This way one can use the default PCRE function -names without getting access violations on runtime. - - <ALLLIB value="libpcre.lib $(LIBFILES) $(LIBRARIES) import32.lib cp32mt.lib"/> - - -BUILDING PCRE ON OPENVMS - -Stephen Hoffman sent the following, in December 2012: - -"Here <http://labs.hoffmanlabs.com/node/1847> is a very short write-up on the -OpenVMS port and here - -<http://labs.hoffmanlabs.com/labsnotes/pcre-vms-8_32.zip> - -is a zip with the OpenVMS files, and with one modified testing-related PCRE -file." This is a port of PCRE 8.32. - -Earlier, Dan Mooney sent the following comments about building PCRE on OpenVMS. -They relate to an older version of PCRE that used fewer source files, so the -exact commands will need changing. See the current list of source files above. - -"It was quite easy to compile and link the library. I don't have a formal -make file but the attached file [reproduced below] contains the OpenVMS DCL -commands I used to build the library. I had to add #define -POSIX_MALLOC_THRESHOLD 10 to pcre.h since it was not defined anywhere. - -The library was built on: -O/S: HP OpenVMS v7.3-1 -Compiler: Compaq C v6.5-001-48BCD -Linker: vA13-01 - -The test results did not match 100% due to the issues you mention in your -documentation regarding isprint(), iscntrl(), isgraph() and ispunct(). I -modified some of the character tables temporarily and was able to get the -results to match. Tests using the fr locale did not match since I don't have -that locale loaded. The study size was always reported to be 3 less than the -value in the standard test output files." - -========================= -$! This DCL procedure builds PCRE on OpenVMS -$! -$! I followed the instructions in the non-unix-use file in the distribution. -$! -$ COMPILE == "CC/LIST/NOMEMBER_ALIGNMENT/PREFIX_LIBRARY_ENTRIES=ALL_ENTRIES -$ COMPILE DFTABLES.C -$ LINK/EXE=DFTABLES.EXE DFTABLES.OBJ -$ RUN DFTABLES.EXE/OUTPUT=CHARTABLES.C -$ COMPILE MAKETABLES.C -$ COMPILE GET.C -$ COMPILE STUDY.C -$! I had to set POSIX_MALLOC_THRESHOLD to 10 in PCRE.H since the symbol -$! did not seem to be defined anywhere. -$! I edited pcre.h and added #DEFINE SUPPORT_UTF8 to enable UTF8 support. -$ COMPILE PCRE.C -$ LIB/CREATE PCRE MAKETABLES.OBJ, GET.OBJ, STUDY.OBJ, PCRE.OBJ -$! I had to set POSIX_MALLOC_THRESHOLD to 10 in PCRE.H since the symbol -$! did not seem to be defined anywhere. -$ COMPILE PCREPOSIX.C -$ LIB/CREATE PCREPOSIX PCREPOSIX.OBJ -$ COMPILE PCRETEST.C -$ LINK/EXE=PCRETEST.EXE PCRETEST.OBJ, PCRE/LIB, PCREPOSIX/LIB -$! C programs that want access to command line arguments must be -$! defined as a symbol -$ PCRETEST :== "$ SYS$ROADSUSERS:[DMOONEY.REGEXP]PCRETEST.EXE" -$! Arguments must be enclosed in quotes. -$ PCRETEST "-C" -$! Test results: -$! -$! The test results did not match 100%. The functions isprint(), iscntrl(), -$! isgraph() and ispunct() on OpenVMS must not produce the same results -$! as the system that built the test output files provided with the -$! distribution. -$! -$! The study size did not match and was always 3 less on OpenVMS. -$! -$! Locale could not be set to fr -$! -========================= - - -BUILDING PCRE ON STRATUS OPENVOS - -These notes on the port of PCRE to VOS (lightly edited) were supplied by -Ashutosh Warikoo, whose email address has the local part awarikoo and the -domain nse.co.in. The port was for version 7.9 in August 2009. - -1. Building PCRE - -I built pcre on OpenVOS Release 17.0.1at using GNU Tools 3.4a without any -problems. I used the following packages to build PCRE: - - ftp://ftp.stratus.com/pub/vos/posix/ga/posix.save.evf.gz - -Please read and follow the instructions that come with these packages. To start -the build of pcre, from the root of the package type: - - ./build.sh - -2. Installing PCRE - -Once you have successfully built PCRE, login to the SysAdmin group, switch to -the root user, and type - - [ !create_dir (master_disk)>usr --if needed ] - [ !create_dir (master_disk)>usr>local --if needed ] - !gmake install - -This installs PCRE and its man pages into /usr/local. You can add -(master_disk)>usr>local>bin to your command search paths, or if you are in -BASH, add /usr/local/bin to the PATH environment variable. - -4. Restrictions - -This port requires readline library optionally. However during the build I -faced some yet unexplored errors while linking with readline. As it was an -optional component I chose to disable it. - -5. Known Problems - -I ran the test suite, but you will have to be your own judge of whether this -command, and this port, suits your purposes. If you find any problems that -appear to be related to the port itself, please let me know. Please see the -build.log file in the root of the package also. - - -BUILDING PCRE ON NATIVE Z/OS AND Z/VM - -z/OS and z/VM are operating systems for mainframe computers, produced by IBM. -The character code used is EBCDIC, not ASCII or Unicode. In z/OS, UNIX APIs and -applications can be supported through UNIX System Services, and in such an -environment PCRE can be built in the same way as in other systems. However, in -native z/OS (without UNIX System Services) and in z/VM, special ports are -required. For details, please see this web site: - - http://www.zaconsultants.net - -There is also a mirror here: - - http://www.vsoft-software.com/downloads.html - -========================== -Last Updated: 14 May 2013 diff --git a/plugins/Pcre16/docs/NON-UNIX-USE b/plugins/Pcre16/docs/NON-UNIX-USE deleted file mode 100644 index a25546b6ff..0000000000 --- a/plugins/Pcre16/docs/NON-UNIX-USE +++ /dev/null @@ -1,7 +0,0 @@ -Compiling PCRE on non-Unix systems ----------------------------------- - -This has been renamed to better reflect its contents. Please see the file -NON-AUTOTOOLS-BUILD for details of how to build PCRE without using autotools. - -#### diff --git a/plugins/Pcre16/docs/README b/plugins/Pcre16/docs/README deleted file mode 100644 index 88f2dfd4ef..0000000000 --- a/plugins/Pcre16/docs/README +++ /dev/null @@ -1,991 +0,0 @@ -README file for PCRE (Perl-compatible regular expression library) ------------------------------------------------------------------ - -The latest release of PCRE is always available in three alternative formats -from: - - ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz - ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.bz2 - ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.zip - -There is a mailing list for discussion about the development of PCRE at -pcre-dev@exim.org. You can access the archives and subscribe or manage your -subscription here: - - https://lists.exim.org/mailman/listinfo/pcre-dev - -Please read the NEWS file if you are upgrading from a previous release. -The contents of this README file are: - - The PCRE APIs - Documentation for PCRE - Contributions by users of PCRE - Building PCRE on non-Unix-like systems - Building PCRE without using autotools - Building PCRE using autotools - Retrieving configuration information - Shared libraries - Cross-compiling using autotools - Using HP's ANSI C++ compiler (aCC) - Compiling in Tru64 using native compilers - Using Sun's compilers for Solaris - Using PCRE from MySQL - Making new tarballs - Testing PCRE - Character tables - File manifest - - -The PCRE APIs -------------- - -PCRE is written in C, and it has its own API. There are three sets of -functions, one for the 8-bit library, which processes strings of bytes, one for -the 16-bit library, which processes strings of 16-bit values, and one for the -32-bit library, which processes strings of 32-bit values. The distribution also -includes a set of C++ wrapper functions (see the pcrecpp man page for details), -courtesy of Google Inc., which can be used to call the 8-bit PCRE library from -C++. - -In addition, there is a set of C wrapper functions (again, just for the 8-bit -library) that are based on the POSIX regular expression API (see the pcreposix -man page). These end up in the library called libpcreposix. Note that this just -provides a POSIX calling interface to PCRE; the regular expressions themselves -still follow Perl syntax and semantics. The POSIX API is restricted, and does -not give full access to all of PCRE's facilities. - -The header file for the POSIX-style functions is called pcreposix.h. The -official POSIX name is regex.h, but I did not want to risk possible problems -with existing files of that name by distributing it that way. To use PCRE with -an existing program that uses the POSIX API, pcreposix.h will have to be -renamed or pointed at by a link. - -If you are using the POSIX interface to PCRE and there is already a POSIX regex -library installed on your system, as well as worrying about the regex.h header -file (as mentioned above), you must also take care when linking programs to -ensure that they link with PCRE's libpcreposix library. Otherwise they may pick -up the POSIX functions of the same name from the other library. - -One way of avoiding this confusion is to compile PCRE with the addition of --Dregcomp=PCREregcomp (and similarly for the other POSIX functions) to the -compiler flags (CFLAGS if you are using "configure" -- see below). This has the -effect of renaming the functions so that the names no longer clash. Of course, -you have to do the same thing for your applications, or write them using the -new names. - - -Documentation for PCRE ----------------------- - -If you install PCRE in the normal way on a Unix-like system, you will end up -with a set of man pages whose names all start with "pcre". The one that is just -called "pcre" lists all the others. In addition to these man pages, the PCRE -documentation is supplied in two other forms: - - 1. There are files called doc/pcre.txt, doc/pcregrep.txt, and - doc/pcretest.txt in the source distribution. The first of these is a - concatenation of the text forms of all the section 3 man pages except - the listing of pcredemo.c and those that summarize individual functions. - The other two are the text forms of the section 1 man pages for the - pcregrep and pcretest commands. These text forms are provided for ease of - scanning with text editors or similar tools. They are installed in - <prefix>/share/doc/pcre, where <prefix> is the installation prefix - (defaulting to /usr/local). - - 2. A set of files containing all the documentation in HTML form, hyperlinked - in various ways, and rooted in a file called index.html, is distributed in - doc/html and installed in <prefix>/share/doc/pcre/html. - -Users of PCRE have contributed files containing the documentation for various -releases in CHM format. These can be found in the Contrib directory of the FTP -site (see next section). - - -Contributions by users of PCRE ------------------------------- - -You can find contributions from PCRE users in the directory - - ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib - -There is a README file giving brief descriptions of what they are. Some are -complete in themselves; others are pointers to URLs containing relevant files. -Some of this material is likely to be well out-of-date. Several of the earlier -contributions provided support for compiling PCRE on various flavours of -Windows (I myself do not use Windows). Nowadays there is more Windows support -in the standard distribution, so these contibutions have been archived. - -A PCRE user maintains downloadable Windows binaries of the pcregrep and -pcretest programs here: - - http://www.rexegg.com/pcregrep-pcretest.html - - -Building PCRE on non-Unix-like systems --------------------------------------- - -For a non-Unix-like system, please read the comments in the file -NON-AUTOTOOLS-BUILD, though if your system supports the use of "configure" and -"make" you may be able to build PCRE using autotools in the same way as for -many Unix-like systems. - -PCRE can also be configured using the GUI facility provided by CMake's -cmake-gui command. This creates Makefiles, solution files, etc. The file -NON-AUTOTOOLS-BUILD has information about CMake. - -PCRE has been compiled on many different operating systems. It should be -straightforward to build PCRE on any system that has a Standard C compiler and -library, because it uses only Standard C functions. - - -Building PCRE without using autotools -------------------------------------- - -The use of autotools (in particular, libtool) is problematic in some -environments, even some that are Unix or Unix-like. See the NON-AUTOTOOLS-BUILD -file for ways of building PCRE without using autotools. - - -Building PCRE using autotools ------------------------------ - -If you are using HP's ANSI C++ compiler (aCC), please see the special note -in the section entitled "Using HP's ANSI C++ compiler (aCC)" below. - -The following instructions assume the use of the widely used "configure; make; -make install" (autotools) process. - -To build PCRE on system that supports autotools, first run the "configure" -command from the PCRE distribution directory, with your current directory set -to the directory where you want the files to be created. This command is a -standard GNU "autoconf" configuration script, for which generic instructions -are supplied in the file INSTALL. - -Most commonly, people build PCRE within its own distribution directory, and in -this case, on many systems, just running "./configure" is sufficient. However, -the usual methods of changing standard defaults are available. For example: - -CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local - -This command specifies that the C compiler should be run with the flags '-O2 --Wall' instead of the default, and that "make install" should install PCRE -under /opt/local instead of the default /usr/local. - -If you want to build in a different directory, just run "configure" with that -directory as current. For example, suppose you have unpacked the PCRE source -into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx: - -cd /build/pcre/pcre-xxx -/source/pcre/pcre-xxx/configure - -PCRE is written in C and is normally compiled as a C library. However, it is -possible to build it as a C++ library, though the provided building apparatus -does not have any features to support this. - -There are some optional features that can be included or omitted from the PCRE -library. They are also documented in the pcrebuild man page. - -. By default, both shared and static libraries are built. You can change this - by adding one of these options to the "configure" command: - - --disable-shared - --disable-static - - (See also "Shared libraries on Unix-like systems" below.) - -. By default, only the 8-bit library is built. If you add --enable-pcre16 to - the "configure" command, the 16-bit library is also built. If you add - --enable-pcre32 to the "configure" command, the 32-bit library is also built. - If you want only the 16-bit or 32-bit library, use --disable-pcre8 to disable - building the 8-bit library. - -. If you are building the 8-bit library and want to suppress the building of - the C++ wrapper library, you can add --disable-cpp to the "configure" - command. Otherwise, when "configure" is run without --disable-pcre8, it will - try to find a C++ compiler and C++ header files, and if it succeeds, it will - try to build the C++ wrapper. - -. If you want to include support for just-in-time compiling, which can give - large performance improvements on certain platforms, add --enable-jit to the - "configure" command. This support is available only for certain hardware - architectures. If you try to enable it on an unsupported architecture, there - will be a compile time error. - -. When JIT support is enabled, pcregrep automatically makes use of it, unless - you add --disable-pcregrep-jit to the "configure" command. - -. If you want to make use of the support for UTF-8 Unicode character strings in - the 8-bit library, or UTF-16 Unicode character strings in the 16-bit library, - or UTF-32 Unicode character strings in the 32-bit library, you must add - --enable-utf to the "configure" command. Without it, the code for handling - UTF-8, UTF-16 and UTF-8 is not included in the relevant library. Even - when --enable-utf is included, the use of a UTF encoding still has to be - enabled by an option at run time. When PCRE is compiled with this option, its - input can only either be ASCII or UTF-8/16/32, even when running on EBCDIC - platforms. It is not possible to use both --enable-utf and --enable-ebcdic at - the same time. - -. There are no separate options for enabling UTF-8, UTF-16 and UTF-32 - independently because that would allow ridiculous settings such as requesting - UTF-16 support while building only the 8-bit library. However, the option - --enable-utf8 is retained for backwards compatibility with earlier releases - that did not support 16-bit or 32-bit character strings. It is synonymous with - --enable-utf. It is not possible to configure one library with UTF support - and the other without in the same configuration. - -. If, in addition to support for UTF-8/16/32 character strings, you want to - include support for the \P, \p, and \X sequences that recognize Unicode - character properties, you must add --enable-unicode-properties to the - "configure" command. This adds about 30K to the size of the library (in the - form of a property table); only the basic two-letter properties such as Lu - are supported. - -. You can build PCRE to recognize either CR or LF or the sequence CRLF or any - of the preceding, or any of the Unicode newline sequences as indicating the - end of a line. Whatever you specify at build time is the default; the caller - of PCRE can change the selection at run time. The default newline indicator - is a single LF character (the Unix standard). You can specify the default - newline indicator by adding --enable-newline-is-cr or --enable-newline-is-lf - or --enable-newline-is-crlf or --enable-newline-is-anycrlf or - --enable-newline-is-any to the "configure" command, respectively. - - If you specify --enable-newline-is-cr or --enable-newline-is-crlf, some of - the standard tests will fail, because the lines in the test files end with - LF. Even if the files are edited to change the line endings, there are likely - to be some failures. With --enable-newline-is-anycrlf or - --enable-newline-is-any, many tests should succeed, but there may be some - failures. - -. By default, the sequence \R in a pattern matches any Unicode line ending - sequence. This is independent of the option specifying what PCRE considers to - be the end of a line (see above). However, the caller of PCRE can restrict \R - to match only CR, LF, or CRLF. You can make this the default by adding - --enable-bsr-anycrlf to the "configure" command (bsr = "backslash R"). - -. When called via the POSIX interface, PCRE uses malloc() to get additional - storage for processing capturing parentheses if there are more than 10 of - them in a pattern. You can increase this threshold by setting, for example, - - --with-posix-malloc-threshold=20 - - on the "configure" command. - -. PCRE has a counter that limits the depth of nesting of parentheses in a - pattern. This limits the amount of system stack that a pattern uses when it - is compiled. The default is 250, but you can change it by setting, for - example, - - --with-parens-nest-limit=500 - -. PCRE has a counter that can be set to limit the amount of resources it uses - when matching a pattern. If the limit is exceeded during a match, the match - fails. The default is ten million. You can change the default by setting, for - example, - - --with-match-limit=500000 - - on the "configure" command. This is just the default; individual calls to - pcre_exec() can supply their own value. There is more discussion on the - pcreapi man page. - -. There is a separate counter that limits the depth of recursive function calls - during a matching process. This also has a default of ten million, which is - essentially "unlimited". You can change the default by setting, for example, - - --with-match-limit-recursion=500000 - - Recursive function calls use up the runtime stack; running out of stack can - cause programs to crash in strange ways. There is a discussion about stack - sizes in the pcrestack man page. - -. The default maximum compiled pattern size is around 64K. You can increase - this by adding --with-link-size=3 to the "configure" command. In the 8-bit - library, PCRE then uses three bytes instead of two for offsets to different - parts of the compiled pattern. In the 16-bit library, --with-link-size=3 is - the same as --with-link-size=4, which (in both libraries) uses four-byte - offsets. Increasing the internal link size reduces performance. In the 32-bit - library, the only supported link size is 4. - -. You can build PCRE so that its internal match() function that is called from - pcre_exec() does not call itself recursively. Instead, it uses memory blocks - obtained from the heap via the special functions pcre_stack_malloc() and - pcre_stack_free() to save data that would otherwise be saved on the stack. To - build PCRE like this, use - - --disable-stack-for-recursion - - on the "configure" command. PCRE runs more slowly in this mode, but it may be - necessary in environments with limited stack sizes. This applies only to the - normal execution of the pcre_exec() function; if JIT support is being - successfully used, it is not relevant. Equally, it does not apply to - pcre_dfa_exec(), which does not use deeply nested recursion. There is a - discussion about stack sizes in the pcrestack man page. - -. For speed, PCRE uses four tables for manipulating and identifying characters - whose code point values are less than 256. By default, it uses a set of - tables for ASCII encoding that is part of the distribution. If you specify - - --enable-rebuild-chartables - - a program called dftables is compiled and run in the default C locale when - you obey "make". It builds a source file called pcre_chartables.c. If you do - not specify this option, pcre_chartables.c is created as a copy of - pcre_chartables.c.dist. See "Character tables" below for further information. - -. It is possible to compile PCRE for use on systems that use EBCDIC as their - character code (as opposed to ASCII/Unicode) by specifying - - --enable-ebcdic - - This automatically implies --enable-rebuild-chartables (see above). However, - when PCRE is built this way, it always operates in EBCDIC. It cannot support - both EBCDIC and UTF-8/16/32. There is a second option, --enable-ebcdic-nl25, - which specifies that the code value for the EBCDIC NL character is 0x25 - instead of the default 0x15. - -. In environments where valgrind is installed, if you specify - - --enable-valgrind - - PCRE will use valgrind annotations to mark certain memory regions as - unaddressable. This allows it to detect invalid memory accesses, and is - mostly useful for debugging PCRE itself. - -. In environments where the gcc compiler is used and lcov version 1.6 or above - is installed, if you specify - - --enable-coverage - - the build process implements a code coverage report for the test suite. The - report is generated by running "make coverage". If ccache is installed on - your system, it must be disabled when building PCRE for coverage reporting. - You can do this by setting the environment variable CCACHE_DISABLE=1 before - running "make" to build PCRE. There is more information about coverage - reporting in the "pcrebuild" documentation. - -. The pcregrep program currently supports only 8-bit data files, and so - requires the 8-bit PCRE library. It is possible to compile pcregrep to use - libz and/or libbz2, in order to read .gz and .bz2 files (respectively), by - specifying one or both of - - --enable-pcregrep-libz - --enable-pcregrep-libbz2 - - Of course, the relevant libraries must be installed on your system. - -. The default size (in bytes) of the internal buffer used by pcregrep can be - set by, for example: - - --with-pcregrep-bufsize=51200 - - The value must be a plain integer. The default is 20480. - -. It is possible to compile pcretest so that it links with the libreadline - or libedit libraries, by specifying, respectively, - - --enable-pcretest-libreadline or --enable-pcretest-libedit - - If this is done, when pcretest's input is from a terminal, it reads it using - the readline() function. This provides line-editing and history facilities. - Note that libreadline is GPL-licenced, so if you distribute a binary of - pcretest linked in this way, there may be licensing issues. These can be - avoided by linking with libedit (which has a BSD licence) instead. - - Enabling libreadline causes the -lreadline option to be added to the pcretest - build. In many operating environments with a sytem-installed readline - library this is sufficient. However, in some environments (e.g. if an - unmodified distribution version of readline is in use), it may be necessary - to specify something like LIBS="-lncurses" as well. This is because, to quote - the readline INSTALL, "Readline uses the termcap functions, but does not link - with the termcap or curses library itself, allowing applications which link - with readline the to choose an appropriate library." If you get error - messages about missing functions tgetstr, tgetent, tputs, tgetflag, or tgoto, - this is the problem, and linking with the ncurses library should fix it. - -The "configure" script builds the following files for the basic C library: - -. Makefile the makefile that builds the library -. config.h build-time configuration options for the library -. pcre.h the public PCRE header file -. pcre-config script that shows the building settings such as CFLAGS - that were set for "configure" -. libpcre.pc ) data for the pkg-config command -. libpcre16.pc ) -. libpcre32.pc ) -. libpcreposix.pc ) -. libtool script that builds shared and/or static libraries - -Versions of config.h and pcre.h are distributed in the PCRE tarballs under the -names config.h.generic and pcre.h.generic. These are provided for those who -have to built PCRE without using "configure" or CMake. If you use "configure" -or CMake, the .generic versions are not used. - -When building the 8-bit library, if a C++ compiler is found, the following -files are also built: - -. libpcrecpp.pc data for the pkg-config command -. pcrecpparg.h header file for calling PCRE via the C++ wrapper -. pcre_stringpiece.h header for the C++ "stringpiece" functions - -The "configure" script also creates config.status, which is an executable -script that can be run to recreate the configuration, and config.log, which -contains compiler output from tests that "configure" runs. - -Once "configure" has run, you can run "make". This builds the the libraries -libpcre, libpcre16 and/or libpcre32, and a test program called pcretest. If you -enabled JIT support with --enable-jit, a test program called pcre_jit_test is -built as well. - -If the 8-bit library is built, libpcreposix and the pcregrep command are also -built, and if a C++ compiler was found on your system, and you did not disable -it with --disable-cpp, "make" builds the C++ wrapper library, which is called -libpcrecpp, as well as some test programs called pcrecpp_unittest, -pcre_scanner_unittest, and pcre_stringpiece_unittest. - -The command "make check" runs all the appropriate tests. Details of the PCRE -tests are given below in a separate section of this document. - -You can use "make install" to install PCRE into live directories on your -system. The following are installed (file names are all relative to the -<prefix> that is set when "configure" is run): - - Commands (bin): - pcretest - pcregrep (if 8-bit support is enabled) - pcre-config - - Libraries (lib): - libpcre16 (if 16-bit support is enabled) - libpcre32 (if 32-bit support is enabled) - libpcre (if 8-bit support is enabled) - libpcreposix (if 8-bit support is enabled) - libpcrecpp (if 8-bit and C++ support is enabled) - - Configuration information (lib/pkgconfig): - libpcre16.pc - libpcre32.pc - libpcre.pc - libpcreposix.pc - libpcrecpp.pc (if C++ support is enabled) - - Header files (include): - pcre.h - pcreposix.h - pcre_scanner.h ) - pcre_stringpiece.h ) if C++ support is enabled - pcrecpp.h ) - pcrecpparg.h ) - - Man pages (share/man/man{1,3}): - pcregrep.1 - pcretest.1 - pcre-config.1 - pcre.3 - pcre*.3 (lots more pages, all starting "pcre") - - HTML documentation (share/doc/pcre/html): - index.html - *.html (lots more pages, hyperlinked from index.html) - - Text file documentation (share/doc/pcre): - AUTHORS - COPYING - ChangeLog - LICENCE - NEWS - README - pcre.txt (a concatenation of the man(3) pages) - pcretest.txt the pcretest man page - pcregrep.txt the pcregrep man page - pcre-config.txt the pcre-config man page - -If you want to remove PCRE from your system, you can run "make uninstall". -This removes all the files that "make install" installed. However, it does not -remove any directories, because these are often shared with other programs. - - -Retrieving configuration information ------------------------------------- - -Running "make install" installs the command pcre-config, which can be used to -recall information about the PCRE configuration and installation. For example: - - pcre-config --version - -prints the version number, and - - pcre-config --libs - -outputs information about where the library is installed. This command can be -included in makefiles for programs that use PCRE, saving the programmer from -having to remember too many details. - -The pkg-config command is another system for saving and retrieving information -about installed libraries. Instead of separate commands for each library, a -single command is used. For example: - - pkg-config --cflags pcre - -The data is held in *.pc files that are installed in a directory called -<prefix>/lib/pkgconfig. - - -Shared libraries ----------------- - -The default distribution builds PCRE as shared libraries and static libraries, -as long as the operating system supports shared libraries. Shared library -support relies on the "libtool" script which is built as part of the -"configure" process. - -The libtool script is used to compile and link both shared and static -libraries. They are placed in a subdirectory called .libs when they are newly -built. The programs pcretest and pcregrep are built to use these uninstalled -libraries (by means of wrapper scripts in the case of shared libraries). When -you use "make install" to install shared libraries, pcregrep and pcretest are -automatically re-built to use the newly installed shared libraries before being -installed themselves. However, the versions left in the build directory still -use the uninstalled libraries. - -To build PCRE using static libraries only you must use --disable-shared when -configuring it. For example: - -./configure --prefix=/usr/gnu --disable-shared - -Then run "make" in the usual way. Similarly, you can use --disable-static to -build only shared libraries. - - -Cross-compiling using autotools -------------------------------- - -You can specify CC and CFLAGS in the normal way to the "configure" command, in -order to cross-compile PCRE for some other host. However, you should NOT -specify --enable-rebuild-chartables, because if you do, the dftables.c source -file is compiled and run on the local host, in order to generate the inbuilt -character tables (the pcre_chartables.c file). This will probably not work, -because dftables.c needs to be compiled with the local compiler, not the cross -compiler. - -When --enable-rebuild-chartables is not specified, pcre_chartables.c is created -by making a copy of pcre_chartables.c.dist, which is a default set of tables -that assumes ASCII code. Cross-compiling with the default tables should not be -a problem. - -If you need to modify the character tables when cross-compiling, you should -move pcre_chartables.c.dist out of the way, then compile dftables.c by hand and -run it on the local host to make a new version of pcre_chartables.c.dist. -Then when you cross-compile PCRE this new version of the tables will be used. - - -Using HP's ANSI C++ compiler (aCC) ----------------------------------- - -Unless C++ support is disabled by specifying the "--disable-cpp" option of the -"configure" script, you must include the "-AA" option in the CXXFLAGS -environment variable in order for the C++ components to compile correctly. - -Also, note that the aCC compiler on PA-RISC platforms may have a defect whereby -needed libraries fail to get included when specifying the "-AA" compiler -option. If you experience unresolved symbols when linking the C++ programs, -use the workaround of specifying the following environment variable prior to -running the "configure" script: - - CXXLDFLAGS="-lstd_v2 -lCsup_v2" - - -Compiling in Tru64 using native compilers ------------------------------------------ - -The following error may occur when compiling with native compilers in the Tru64 -operating system: - - CXX libpcrecpp_la-pcrecpp.lo -cxx: Error: /usr/lib/cmplrs/cxx/V7.1-006/include/cxx/iosfwd, line 58: #error - directive: "cannot include iosfwd -- define __USE_STD_IOSTREAM to - override default - see section 7.1.2 of the C++ Using Guide" -#error "cannot include iosfwd -- define __USE_STD_IOSTREAM to override default -- see section 7.1.2 of the C++ Using Guide" - -This may be followed by other errors, complaining that 'namespace "std" has no -member'. The solution to this is to add the line - -#define __USE_STD_IOSTREAM 1 - -to the config.h file. - - -Using Sun's compilers for Solaris ---------------------------------- - -A user reports that the following configurations work on Solaris 9 sparcv9 and -Solaris 9 x86 (32-bit): - - Solaris 9 sparcv9: ./configure --disable-cpp CC=/bin/cc CFLAGS="-m64 -g" - Solaris 9 x86: ./configure --disable-cpp CC=/bin/cc CFLAGS="-g" - - -Using PCRE from MySQL ---------------------- - -On systems where both PCRE and MySQL are installed, it is possible to make use -of PCRE from within MySQL, as an alternative to the built-in pattern matching. -There is a web page that tells you how to do this: - - http://www.mysqludf.org/lib_mysqludf_preg/index.php - - -Making new tarballs -------------------- - -The command "make dist" creates three PCRE tarballs, in tar.gz, tar.bz2, and -zip formats. The command "make distcheck" does the same, but then does a trial -build of the new distribution to ensure that it works. - -If you have modified any of the man page sources in the doc directory, you -should first run the PrepareRelease script before making a distribution. This -script creates the .txt and HTML forms of the documentation from the man pages. - - -Testing PCRE ------------- - -To test the basic PCRE library on a Unix-like system, run the RunTest script. -There is another script called RunGrepTest that tests the options of the -pcregrep command. If the C++ wrapper library is built, three test programs -called pcrecpp_unittest, pcre_scanner_unittest, and pcre_stringpiece_unittest -are also built. When JIT support is enabled, another test program called -pcre_jit_test is built. - -Both the scripts and all the program tests are run if you obey "make check" or -"make test". For other environments, see the instructions in -NON-AUTOTOOLS-BUILD. - -The RunTest script runs the pcretest test program (which is documented in its -own man page) on each of the relevant testinput files in the testdata -directory, and compares the output with the contents of the corresponding -testoutput files. RunTest uses a file called testtry to hold the main output -from pcretest. Other files whose names begin with "test" are used as working -files in some tests. - -Some tests are relevant only when certain build-time options were selected. For -example, the tests for UTF-8/16/32 support are run only if --enable-utf was -used. RunTest outputs a comment when it skips a test. - -Many of the tests that are not skipped are run up to three times. The second -run forces pcre_study() to be called for all patterns except for a few in some -tests that are marked "never study" (see the pcretest program for how this is -done). If JIT support is available, the non-DFA tests are run a third time, -this time with a forced pcre_study() with the PCRE_STUDY_JIT_COMPILE option. -This testing can be suppressed by putting "nojit" on the RunTest command line. - -The entire set of tests is run once for each of the 8-bit, 16-bit and 32-bit -libraries that are enabled. If you want to run just one set of tests, call -RunTest with either the -8, -16 or -32 option. - -If valgrind is installed, you can run the tests under it by putting "valgrind" -on the RunTest command line. To run pcretest on just one or more specific test -files, give their numbers as arguments to RunTest, for example: - - RunTest 2 7 11 - -You can also specify ranges of tests such as 3-6 or 3- (meaning 3 to the -end), or a number preceded by ~ to exclude a test. For example: - - Runtest 3-15 ~10 - -This runs tests 3 to 15, excluding test 10, and just ~13 runs all the tests -except test 13. Whatever order the arguments are in, the tests are always run -in numerical order. - -You can also call RunTest with the single argument "list" to cause it to output -a list of tests. - -The first test file can be fed directly into the perltest.pl script to check -that Perl gives the same results. The only difference you should see is in the -first few lines, where the Perl version is given instead of the PCRE version. - -The second set of tests check pcre_fullinfo(), pcre_study(), -pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error -detection, and run-time flags that are specific to PCRE, as well as the POSIX -wrapper API. It also uses the debugging flags to check some of the internals of -pcre_compile(). - -If you build PCRE with a locale setting that is not the standard C locale, the -character tables may be different (see next paragraph). In some cases, this may -cause failures in the second set of tests. For example, in a locale where the -isprint() function yields TRUE for characters in the range 128-255, the use of -[:isascii:] inside a character class defines a different set of characters, and -this shows up in this test as a difference in the compiled code, which is being -listed for checking. Where the comparison test output contains [\x00-\x7f] the -test will contain [\x00-\xff], and similarly in some other cases. This is not a -bug in PCRE. - -The third set of tests checks pcre_maketables(), the facility for building a -set of character tables for a specific locale and using them instead of the -default tables. The tests make use of the "fr_FR" (French) locale. Before -running the test, the script checks for the presence of this locale by running -the "locale" command. If that command fails, or if it doesn't include "fr_FR" -in the list of available locales, the third test cannot be run, and a comment -is output to say why. If running this test produces instances of the error - - ** Failed to set locale "fr_FR" - -in the comparison output, it means that locale is not available on your system, -despite being listed by "locale". This does not mean that PCRE is broken. - -[If you are trying to run this test on Windows, you may be able to get it to -work by changing "fr_FR" to "french" everywhere it occurs. Alternatively, use -RunTest.bat. The version of RunTest.bat included with PCRE 7.4 and above uses -Windows versions of test 2. More info on using RunTest.bat is included in the -document entitled NON-UNIX-USE.] - -The fourth and fifth tests check the UTF-8/16/32 support and error handling and -internal UTF features of PCRE that are not relevant to Perl, respectively. The -sixth and seventh tests do the same for Unicode character properties support. - -The eighth, ninth, and tenth tests check the pcre_dfa_exec() alternative -matching function, in non-UTF-8/16/32 mode, UTF-8/16/32 mode, and UTF-8/16/32 -mode with Unicode property support, respectively. - -The eleventh test checks some internal offsets and code size features; it is -run only when the default "link size" of 2 is set (in other cases the sizes -change) and when Unicode property support is enabled. - -The twelfth test is run only when JIT support is available, and the thirteenth -test is run only when JIT support is not available. They test some JIT-specific -features such as information output from pcretest about JIT compilation. - -The fourteenth, fifteenth, and sixteenth tests are run only in 8-bit mode, and -the seventeenth, eighteenth, and nineteenth tests are run only in 16/32-bit -mode. These are tests that generate different output in the two modes. They are -for general cases, UTF-8/16/32 support, and Unicode property support, -respectively. - -The twentieth test is run only in 16/32-bit mode. It tests some specific -16/32-bit features of the DFA matching engine. - -The twenty-first and twenty-second tests are run only in 16/32-bit mode, when -the link size is set to 2 for the 16-bit library. They test reloading -pre-compiled patterns. - -The twenty-third and twenty-fourth tests are run only in 16-bit mode. They are -for general cases, and UTF-16 support, respectively. - -The twenty-fifth and twenty-sixth tests are run only in 32-bit mode. They are -for general cases, and UTF-32 support, respectively. - - -Character tables ----------------- - -For speed, PCRE uses four tables for manipulating and identifying characters -whose code point values are less than 256. The final argument of the -pcre_compile() function is a pointer to a block of memory containing the -concatenated tables. A call to pcre_maketables() can be used to generate a set -of tables in the current locale. If the final argument for pcre_compile() is -passed as NULL, a set of default tables that is built into the binary is used. - -The source file called pcre_chartables.c contains the default set of tables. By -default, this is created as a copy of pcre_chartables.c.dist, which contains -tables for ASCII coding. However, if --enable-rebuild-chartables is specified -for ./configure, a different version of pcre_chartables.c is built by the -program dftables (compiled from dftables.c), which uses the ANSI C character -handling functions such as isalnum(), isalpha(), isupper(), islower(), etc. to -build the table sources. This means that the default C locale which is set for -your system will control the contents of these default tables. You can change -the default tables by editing pcre_chartables.c and then re-building PCRE. If -you do this, you should take care to ensure that the file does not get -automatically re-generated. The best way to do this is to move -pcre_chartables.c.dist out of the way and replace it with your customized -tables. - -When the dftables program is run as a result of --enable-rebuild-chartables, -it uses the default C locale that is set on your system. It does not pay -attention to the LC_xxx environment variables. In other words, it uses the -system's default locale rather than whatever the compiling user happens to have -set. If you really do want to build a source set of character tables in a -locale that is specified by the LC_xxx variables, you can run the dftables -program by hand with the -L option. For example: - - ./dftables -L pcre_chartables.c.special - -The first two 256-byte tables provide lower casing and case flipping functions, -respectively. The next table consists of three 32-byte bit maps which identify -digits, "word" characters, and white space, respectively. These are used when -building 32-byte bit maps that represent character classes for code points less -than 256. - -The final 256-byte table has bits indicating various character types, as -follows: - - 1 white space character - 2 letter - 4 decimal digit - 8 hexadecimal digit - 16 alphanumeric or '_' - 128 regular expression metacharacter or binary zero - -You should not alter the set of characters that contain the 128 bit, as that -will cause PCRE to malfunction. - - -File manifest -------------- - -The distribution should contain the files listed below. Where a file name is -given as pcre[16|32]_xxx it means that there are three files, one with the name -pcre_xxx, one with the name pcre16_xx, and a third with the name pcre32_xxx. - -(A) Source files of the PCRE library functions and their headers: - - dftables.c auxiliary program for building pcre_chartables.c - when --enable-rebuild-chartables is specified - - pcre_chartables.c.dist a default set of character tables that assume ASCII - coding; used, unless --enable-rebuild-chartables is - specified, by copying to pcre[16]_chartables.c - - pcreposix.c ) - pcre[16|32]_byte_order.c ) - pcre[16|32]_compile.c ) - pcre[16|32]_config.c ) - pcre[16|32]_dfa_exec.c ) - pcre[16|32]_exec.c ) - pcre[16|32]_fullinfo.c ) - pcre[16|32]_get.c ) sources for the functions in the library, - pcre[16|32]_globals.c ) and some internal functions that they use - pcre[16|32]_jit_compile.c ) - pcre[16|32]_maketables.c ) - pcre[16|32]_newline.c ) - pcre[16|32]_refcount.c ) - pcre[16|32]_string_utils.c ) - pcre[16|32]_study.c ) - pcre[16|32]_tables.c ) - pcre[16|32]_ucd.c ) - pcre[16|32]_version.c ) - pcre[16|32]_xclass.c ) - pcre_ord2utf8.c ) - pcre_valid_utf8.c ) - pcre16_ord2utf16.c ) - pcre16_utf16_utils.c ) - pcre16_valid_utf16.c ) - pcre32_utf32_utils.c ) - pcre32_valid_utf32.c ) - - pcre[16|32]_printint.c ) debugging function that is used by pcretest, - ) and can also be #included in pcre_compile() - - pcre.h.in template for pcre.h when built by "configure" - pcreposix.h header for the external POSIX wrapper API - pcre_internal.h header for internal use - sljit/* 16 files that make up the JIT compiler - ucp.h header for Unicode property handling - - config.h.in template for config.h, which is built by "configure" - - pcrecpp.h public header file for the C++ wrapper - pcrecpparg.h.in template for another C++ header file - pcre_scanner.h public header file for C++ scanner functions - pcrecpp.cc ) - pcre_scanner.cc ) source for the C++ wrapper library - - pcre_stringpiece.h.in template for pcre_stringpiece.h, the header for the - C++ stringpiece functions - pcre_stringpiece.cc source for the C++ stringpiece functions - -(B) Source files for programs that use PCRE: - - pcredemo.c simple demonstration of coding calls to PCRE - pcregrep.c source of a grep utility that uses PCRE - pcretest.c comprehensive test program - -(C) Auxiliary files: - - 132html script to turn "man" pages into HTML - AUTHORS information about the author of PCRE - ChangeLog log of changes to the code - CleanTxt script to clean nroff output for txt man pages - Detrail script to remove trailing spaces - HACKING some notes about the internals of PCRE - INSTALL generic installation instructions - LICENCE conditions for the use of PCRE - COPYING the same, using GNU's standard name - Makefile.in ) template for Unix Makefile, which is built by - ) "configure" - Makefile.am ) the automake input that was used to create - ) Makefile.in - NEWS important changes in this release - NON-UNIX-USE the previous name for NON-AUTOTOOLS-BUILD - NON-AUTOTOOLS-BUILD notes on building PCRE without using autotools - PrepareRelease script to make preparations for "make dist" - README this file - RunTest a Unix shell script for running tests - RunGrepTest a Unix shell script for pcregrep tests - aclocal.m4 m4 macros (generated by "aclocal") - config.guess ) files used by libtool, - config.sub ) used only when building a shared library - configure a configuring shell script (built by autoconf) - configure.ac ) the autoconf input that was used to build - ) "configure" and config.h - depcomp ) script to find program dependencies, generated by - ) automake - doc/*.3 man page sources for PCRE - doc/*.1 man page sources for pcregrep and pcretest - doc/index.html.src the base HTML page - doc/html/* HTML documentation - doc/pcre.txt plain text version of the man pages - doc/pcretest.txt plain text documentation of test program - doc/perltest.txt plain text documentation of Perl test program - install-sh a shell script for installing files - libpcre16.pc.in template for libpcre16.pc for pkg-config - libpcre32.pc.in template for libpcre32.pc for pkg-config - libpcre.pc.in template for libpcre.pc for pkg-config - libpcreposix.pc.in template for libpcreposix.pc for pkg-config - libpcrecpp.pc.in template for libpcrecpp.pc for pkg-config - ltmain.sh file used to build a libtool script - missing ) common stub for a few missing GNU programs while - ) installing, generated by automake - mkinstalldirs script for making install directories - perltest.pl Perl test program - pcre-config.in source of script which retains PCRE information - pcre_jit_test.c test program for the JIT compiler - pcrecpp_unittest.cc ) - pcre_scanner_unittest.cc ) test programs for the C++ wrapper - pcre_stringpiece_unittest.cc ) - testdata/testinput* test data for main library tests - testdata/testoutput* expected test results - testdata/grep* input and output for pcregrep tests - testdata/* other supporting test files - -(D) Auxiliary files for cmake support - - cmake/COPYING-CMAKE-SCRIPTS - cmake/FindPackageHandleStandardArgs.cmake - cmake/FindEditline.cmake - cmake/FindReadline.cmake - CMakeLists.txt - config-cmake.h.in - -(E) Auxiliary files for VPASCAL - - makevp.bat - makevp_c.txt - makevp_l.txt - pcregexp.pas - -(F) Auxiliary files for building PCRE "by hand" - - pcre.h.generic ) a version of the public PCRE header file - ) for use in non-"configure" environments - config.h.generic ) a version of config.h for use in non-"configure" - ) environments - -(F) Miscellaneous - - RunTest.bat a script for running tests under Windows - -Philip Hazel -Email local part: ph10 -Email domain: cam.ac.uk -Last updated: 17 January 2014 diff --git a/plugins/Pcre16/docs/doc/html/NON-AUTOTOOLS-BUILD.txt b/plugins/Pcre16/docs/doc/html/NON-AUTOTOOLS-BUILD.txt deleted file mode 100644 index cddf3e0699..0000000000 --- a/plugins/Pcre16/docs/doc/html/NON-AUTOTOOLS-BUILD.txt +++ /dev/null @@ -1,764 +0,0 @@ -Building PCRE without using autotools -------------------------------------- - -This document contains the following sections: - - General - Generic instructions for the PCRE C library - The C++ wrapper functions - Building for virtual Pascal - Stack size in Windows environments - Linking programs in Windows environments - Calling conventions in Windows environments - Comments about Win32 builds - Building PCRE on Windows with CMake - Use of relative paths with CMake on Windows - Testing with RunTest.bat - Building under Windows CE with Visual Studio 200x - Building under Windows with BCC5.5 - Building using Borland C++ Builder 2007 (CB2007) and higher - Building PCRE on OpenVMS - Building PCRE on Stratus OpenVOS - Building PCRE on native z/OS and z/VM - - -GENERAL - -I (Philip Hazel) have no experience of Windows or VMS sytems and how their -libraries work. The items in the PCRE distribution and Makefile that relate to -anything other than Linux systems are untested by me. - -There are some other comments and files (including some documentation in CHM -format) in the Contrib directory on the FTP site: - - ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib - -The basic PCRE library consists entirely of code written in Standard C, and so -should compile successfully on any system that has a Standard C compiler and -library. The C++ wrapper functions are a separate issue (see below). - -The PCRE distribution includes a "configure" file for use by the configure/make -(autotools) build system, as found in many Unix-like environments. The README -file contains information about the options for "configure". - -There is also support for CMake, which some users prefer, especially in Windows -environments, though it can also be run in Unix-like environments. See the -section entitled "Building PCRE on Windows with CMake" below. - -Versions of config.h and pcre.h are distributed in the PCRE tarballs under the -names config.h.generic and pcre.h.generic. These are provided for those who -build PCRE without using "configure" or CMake. If you use "configure" or CMake, -the .generic versions are not used. - - -GENERIC INSTRUCTIONS FOR THE PCRE C LIBRARY - -The following are generic instructions for building the PCRE C library "by -hand". If you are going to use CMake, this section does not apply to you; you -can skip ahead to the CMake section. - - (1) Copy or rename the file config.h.generic as config.h, and edit the macro - settings that it contains to whatever is appropriate for your environment. - - In particular, you can alter the definition of the NEWLINE macro to - specify what character(s) you want to be interpreted as line terminators. - In an EBCDIC environment, you MUST change NEWLINE, because its default - value is 10, an ASCII LF. The usual EBCDIC newline character is 21 (0x15, - NL), though in some cases it may be 37 (0x25). - - When you compile any of the PCRE modules, you must specify -DHAVE_CONFIG_H - to your compiler so that config.h is included in the sources. - - An alternative approach is not to edit config.h, but to use -D on the - compiler command line to make any changes that you need to the - configuration options. In this case -DHAVE_CONFIG_H must not be set. - - NOTE: There have been occasions when the way in which certain parameters - in config.h are used has changed between releases. (In the configure/make - world, this is handled automatically.) When upgrading to a new release, - you are strongly advised to review config.h.generic before re-using what - you had previously. - - (2) Copy or rename the file pcre.h.generic as pcre.h. - - (3) EITHER: - Copy or rename file pcre_chartables.c.dist as pcre_chartables.c. - - OR: - Compile dftables.c as a stand-alone program (using -DHAVE_CONFIG_H if - you have set up config.h), and then run it with the single argument - "pcre_chartables.c". This generates a set of standard character tables - and writes them to that file. The tables are generated using the default - C locale for your system. If you want to use a locale that is specified - by LC_xxx environment variables, add the -L option to the dftables - command. You must use this method if you are building on a system that - uses EBCDIC code. - - The tables in pcre_chartables.c are defaults. The caller of PCRE can - specify alternative tables at run time. - - (4) Ensure that you have the following header files: - - pcre_internal.h - ucp.h - - (5) For an 8-bit library, compile the following source files, setting - -DHAVE_CONFIG_H as a compiler option if you have set up config.h with your - configuration, or else use other -D settings to change the configuration - as required. - - pcre_byte_order.c - pcre_chartables.c - pcre_compile.c - pcre_config.c - pcre_dfa_exec.c - pcre_exec.c - pcre_fullinfo.c - pcre_get.c - pcre_globals.c - pcre_jit_compile.c - pcre_maketables.c - pcre_newline.c - pcre_ord2utf8.c - pcre_refcount.c - pcre_string_utils.c - pcre_study.c - pcre_tables.c - pcre_ucd.c - pcre_valid_utf8.c - pcre_version.c - pcre_xclass.c - - Make sure that you include -I. in the compiler command (or equivalent for - an unusual compiler) so that all included PCRE header files are first - sought in the current directory. Otherwise you run the risk of picking up - a previously-installed file from somewhere else. - - Note that you must still compile pcre_jit_compile.c, even if you have not - defined SUPPORT_JIT in config.h, because when JIT support is not - configured, dummy functions are compiled. When JIT support IS configured, - pcre_jit_compile.c #includes sources from the sljit subdirectory, where - there should be 16 files, all of whose names begin with "sljit". - - (6) Now link all the compiled code into an object library in whichever form - your system keeps such libraries. This is the basic PCRE C 8-bit library. - If your system has static and shared libraries, you may have to do this - once for each type. - - (7) If you want to build a 16-bit library (as well as, or instead of the 8-bit - or 32-bit libraries) repeat steps 5-6 with the following files: - - pcre16_byte_order.c - pcre16_chartables.c - pcre16_compile.c - pcre16_config.c - pcre16_dfa_exec.c - pcre16_exec.c - pcre16_fullinfo.c - pcre16_get.c - pcre16_globals.c - pcre16_jit_compile.c - pcre16_maketables.c - pcre16_newline.c - pcre16_ord2utf16.c - pcre16_refcount.c - pcre16_string_utils.c - pcre16_study.c - pcre16_tables.c - pcre16_ucd.c - pcre16_utf16_utils.c - pcre16_valid_utf16.c - pcre16_version.c - pcre16_xclass.c - - (8) If you want to build a 32-bit library (as well as, or instead of the 8-bit - or 16-bit libraries) repeat steps 5-6 with the following files: - - pcre32_byte_order.c - pcre32_chartables.c - pcre32_compile.c - pcre32_config.c - pcre32_dfa_exec.c - pcre32_exec.c - pcre32_fullinfo.c - pcre32_get.c - pcre32_globals.c - pcre32_jit_compile.c - pcre32_maketables.c - pcre32_newline.c - pcre32_ord2utf32.c - pcre32_refcount.c - pcre32_string_utils.c - pcre32_study.c - pcre32_tables.c - pcre32_ucd.c - pcre32_utf32_utils.c - pcre32_valid_utf32.c - pcre32_version.c - pcre32_xclass.c - - (9) If you want to build the POSIX wrapper functions (which apply only to the - 8-bit library), ensure that you have the pcreposix.h file and then compile - pcreposix.c (remembering -DHAVE_CONFIG_H if necessary). Link the result - (on its own) as the pcreposix library. - -(10) The pcretest program can be linked with any combination of the 8-bit, - 16-bit and 32-bit libraries (depending on what you selected in config.h). - Compile pcretest.c and pcre_printint.c (again, don't forget - -DHAVE_CONFIG_H) and link them together with the appropriate library/ies. - If you compiled an 8-bit library, pcretest also needs the pcreposix - wrapper library unless you compiled it with -DNOPOSIX. - -(11) Run pcretest on the testinput files in the testdata directory, and check - that the output matches the corresponding testoutput files. There are - comments about what each test does in the section entitled "Testing PCRE" - in the README file. If you compiled more than one of the 8-bit, 16-bit and - 32-bit libraries, you need to run pcretest with the -16 option to do - 16-bit tests and with the -32 option to do 32-bit tests. - - Some tests are relevant only when certain build-time options are selected. - For example, test 4 is for UTF-8/UTF-16/UTF-32 support, and will not run - if you have built PCRE without it. See the comments at the start of each - testinput file. If you have a suitable Unix-like shell, the RunTest script - will run the appropriate tests for you. The command "RunTest list" will - output a list of all the tests. - - Note that the supplied files are in Unix format, with just LF characters - as line terminators. You may need to edit them to change this if your - system uses a different convention. If you are using Windows, you probably - should use the wintestinput3 file instead of testinput3 (and the - corresponding output file). This is a locale test; wintestinput3 sets the - locale to "french" rather than "fr_FR", and there some minor output - differences. - -(12) If you have built PCRE with SUPPORT_JIT, the JIT features will be tested - by the testdata files. However, you might also like to build and run - the freestanding JIT test program, pcre_jit_test.c. - -(13) If you want to use the pcregrep command, compile and link pcregrep.c; it - uses only the basic 8-bit PCRE library (it does not need the pcreposix - library). - - -THE C++ WRAPPER FUNCTIONS - -The PCRE distribution also contains some C++ wrapper functions and tests, -applicable to the 8-bit library, which were contributed by Google Inc. On a -system that can use "configure" and "make", the functions are automatically -built into a library called pcrecpp. It should be straightforward to compile -the .cc files manually on other systems. The files called xxx_unittest.cc are -test programs for each of the corresponding xxx.cc files. - - -BUILDING FOR VIRTUAL PASCAL - -A script for building PCRE using Borland's C++ compiler for use with VPASCAL -was contributed by Alexander Tokarev. Stefan Weber updated the script and added -additional files. The following files in the distribution are for building PCRE -for use with VP/Borland: makevp_c.txt, makevp_l.txt, makevp.bat, pcregexp.pas. - - -STACK SIZE IN WINDOWS ENVIRONMENTS - -The default processor stack size of 1Mb in some Windows environments is too -small for matching patterns that need much recursion. In particular, test 2 may -fail because of this. Normally, running out of stack causes a crash, but there -have been cases where the test program has just died silently. See your linker -documentation for how to increase stack size if you experience problems. The -Linux default of 8Mb is a reasonable choice for the stack, though even that can -be too small for some pattern/subject combinations. - -PCRE has a compile configuration option to disable the use of stack for -recursion so that heap is used instead. However, pattern matching is -significantly slower when this is done. There is more about stack usage in the -"pcrestack" documentation. - - -LINKING PROGRAMS IN WINDOWS ENVIRONMENTS - -If you want to statically link a program against a PCRE library in the form of -a non-dll .a file, you must define PCRE_STATIC before including pcre.h or -pcrecpp.h, otherwise the pcre_malloc() and pcre_free() exported functions will -be declared __declspec(dllimport), with unwanted results. - - -CALLING CONVENTIONS IN WINDOWS ENVIRONMENTS - -It is possible to compile programs to use different calling conventions using -MSVC. Search the web for "calling conventions" for more information. To make it -easier to change the calling convention for the exported functions in the -PCRE library, the macro PCRE_CALL_CONVENTION is present in all the external -definitions. It can be set externally when compiling (e.g. in CFLAGS). If it is -not set, it defaults to empty; the default calling convention is then used -(which is what is wanted most of the time). - - -COMMENTS ABOUT WIN32 BUILDS (see also "BUILDING PCRE ON WINDOWS WITH CMAKE") - -There are two ways of building PCRE using the "configure, make, make install" -paradigm on Windows systems: using MinGW or using Cygwin. These are not at all -the same thing; they are completely different from each other. There is also -support for building using CMake, which some users find a more straightforward -way of building PCRE under Windows. - -The MinGW home page (http://www.mingw.org/) says this: - - MinGW: A collection of freely available and freely distributable Windows - specific header files and import libraries combined with GNU toolsets that - allow one to produce native Windows programs that do not rely on any - 3rd-party C runtime DLLs. - -The Cygwin home page (http://www.cygwin.com/) says this: - - Cygwin is a Linux-like environment for Windows. It consists of two parts: - - . A DLL (cygwin1.dll) which acts as a Linux API emulation layer providing - substantial Linux API functionality - - . A collection of tools which provide Linux look and feel. - - The Cygwin DLL currently works with all recent, commercially released x86 32 - bit and 64 bit versions of Windows, with the exception of Windows CE. - -On both MinGW and Cygwin, PCRE should build correctly using: - - ./configure && make && make install - -This should create two libraries called libpcre and libpcreposix, and, if you -have enabled building the C++ wrapper, a third one called libpcrecpp. These are -independent libraries: when you link with libpcreposix or libpcrecpp you must -also link with libpcre, which contains the basic functions. (Some earlier -releases of PCRE included the basic libpcre functions in libpcreposix. This no -longer happens.) - -A user submitted a special-purpose patch that makes it easy to create -"pcre.dll" under mingw32 using the "msys" environment. It provides "pcre.dll" -as a special target. If you use this target, no other files are built, and in -particular, the pcretest and pcregrep programs are not built. An example of how -this might be used is: - - ./configure --enable-utf --disable-cpp CFLAGS="-03 -s"; make pcre.dll - -Using Cygwin's compiler generates libraries and executables that depend on -cygwin1.dll. If a library that is generated this way is distributed, -cygwin1.dll has to be distributed as well. Since cygwin1.dll is under the GPL -licence, this forces not only PCRE to be under the GPL, but also the entire -application. A distributor who wants to keep their own code proprietary must -purchase an appropriate Cygwin licence. - -MinGW has no such restrictions. The MinGW compiler generates a library or -executable that can run standalone on Windows without any third party dll or -licensing issues. - -But there is more complication: - -If a Cygwin user uses the -mno-cygwin Cygwin gcc flag, what that really does is -to tell Cygwin's gcc to use the MinGW gcc. Cygwin's gcc is only acting as a -front end to MinGW's gcc (if you install Cygwin's gcc, you get both Cygwin's -gcc and MinGW's gcc). So, a user can: - -. Build native binaries by using MinGW or by getting Cygwin and using - -mno-cygwin. - -. Build binaries that depend on cygwin1.dll by using Cygwin with the normal - compiler flags. - -The test files that are supplied with PCRE are in UNIX format, with LF -characters as line terminators. Unless your PCRE library uses a default newline -option that includes LF as a valid newline, it may be necessary to change the -line terminators in the test files to get some of the tests to work. - - -BUILDING PCRE ON WINDOWS WITH CMAKE - -CMake is an alternative configuration facility that can be used instead of -"configure". CMake creates project files (make files, solution files, etc.) -tailored to numerous development environments, including Visual Studio, -Borland, Msys, MinGW, NMake, and Unix. If possible, use short paths with no -spaces in the names for your CMake installation and your PCRE source and build -directories. - -The following instructions were contributed by a PCRE user. If they are not -followed exactly, errors may occur. In the event that errors do occur, it is -recommended that you delete the CMake cache before attempting to repeat the -CMake build process. In the CMake GUI, the cache can be deleted by selecting -"File > Delete Cache". - -1. Install the latest CMake version available from http://www.cmake.org/, and - ensure that cmake\bin is on your path. - -2. Unzip (retaining folder structure) the PCRE source tree into a source - directory such as C:\pcre. You should ensure your local date and time - is not earlier than the file dates in your source dir if the release is - very new. - -3. Create a new, empty build directory, preferably a subdirectory of the - source dir. For example, C:\pcre\pcre-xx\build. - -4. Run cmake-gui from the Shell envirornment of your build tool, for example, - Msys for Msys/MinGW or Visual Studio Command Prompt for VC/VC++. Do not try - to start Cmake from the Windows Start menu, as this can lead to errors. - -5. Enter C:\pcre\pcre-xx and C:\pcre\pcre-xx\build for the source and build - directories, respectively. - -6. Hit the "Configure" button. - -7. Select the particular IDE / build tool that you are using (Visual - Studio, MSYS makefiles, MinGW makefiles, etc.) - -8. The GUI will then list several configuration options. This is where - you can enable UTF-8 support or other PCRE optional features. - -9. Hit "Configure" again. The adjacent "Generate" button should now be - active. - -10. Hit "Generate". - -11. The build directory should now contain a usable build system, be it a - solution file for Visual Studio, makefiles for MinGW, etc. Exit from - cmake-gui and use the generated build system with your compiler or IDE. - E.g., for MinGW you can run "make", or for Visual Studio, open the PCRE - solution, select the desired configuration (Debug, or Release, etc.) and - build the ALL_BUILD project. - -12. If during configuration with cmake-gui you've elected to build the test - programs, you can execute them by building the test project. E.g., for - MinGW: "make test"; for Visual Studio build the RUN_TESTS project. The - most recent build configuration is targeted by the tests. A summary of - test results is presented. Complete test output is subsequently - available for review in Testing\Temporary under your build dir. - - -USE OF RELATIVE PATHS WITH CMAKE ON WINDOWS - -A PCRE user comments as follows: I thought that others may want to know the -current state of CMAKE_USE_RELATIVE_PATHS support on Windows. Here it is: - --- AdditionalIncludeDirectories is only partially modified (only the - first path - see below) --- Only some of the contained file paths are modified - shown below for - pcre.vcproj --- It properly modifies - -I am sure CMake people can fix that if they want to. Until then one will -need to replace existing absolute paths in project files with relative -paths manually (e.g. from VS) - relative to project file location. I did -just that before being told to try CMAKE_USE_RELATIVE_PATHS. Not a big -deal. - -AdditionalIncludeDirectories="E:\builds\pcre\build;E:\builds\pcre\pcre-7.5;" -AdditionalIncludeDirectories=".;E:\builds\pcre\pcre-7.5;" - -RelativePath="pcre.h" -RelativePath="pcre_chartables.c" -RelativePath="pcre_chartables.c.rule" - - -TESTING WITH RUNTEST.BAT - -If configured with CMake, building the test project ("make test" or building -ALL_TESTS in Visual Studio) creates (and runs) pcre_test.bat (and depending -on your configuration options, possibly other test programs) in the build -directory. Pcre_test.bat runs RunTest.Bat with correct source and exe paths. - -For manual testing with RunTest.bat, provided the build dir is a subdirectory -of the source directory: Open command shell window. Chdir to the location -of your pcretest.exe and pcregrep.exe programs. Call RunTest.bat with -"..\RunTest.Bat" or "..\..\RunTest.bat" as appropriate. - -To run only a particular test with RunTest.Bat provide a test number argument. - -Otherwise: - -1. Copy RunTest.bat into the directory where pcretest.exe and pcregrep.exe - have been created. - -2. Edit RunTest.bat to indentify the full or relative location of - the pcre source (wherein which the testdata folder resides), e.g.: - - set srcdir=C:\pcre\pcre-8.20 - -3. In a Windows command environment, chdir to the location of your bat and - exe programs. - -4. Run RunTest.bat. Test outputs will automatically be compared to expected - results, and discrepancies will be identified in the console output. - -To independently test the just-in-time compiler, run pcre_jit_test.exe. -To test pcrecpp, run pcrecpp_unittest.exe, pcre_stringpiece_unittest.exe and -pcre_scanner_unittest.exe. - - -BUILDING UNDER WINDOWS CE WITH VISUAL STUDIO 200x - -Vincent Richomme sent a zip archive of files to help with this process. They -can be found in the file "pcre-vsbuild.zip" in the Contrib directory of the FTP -site. - - -BUILDING UNDER WINDOWS WITH BCC5.5 - -Michael Roy sent these comments about building PCRE under Windows with BCC5.5: - -Some of the core BCC libraries have a version of PCRE from 1998 built in, which -can lead to pcre_exec() giving an erroneous PCRE_ERROR_NULL from a version -mismatch. I'm including an easy workaround below, if you'd like to include it -in the non-unix instructions: - -When linking a project with BCC5.5, pcre.lib must be included before any of the -libraries cw32.lib, cw32i.lib, cw32mt.lib, and cw32mti.lib on the command line. - - -BUILDING USING BORLAND C++ BUILDER 2007 (CB2007) AND HIGHER - -A PCRE user sent these comments about this environment (see also the comment -from another user that follows them): - -The XE versions of C++ Builder come with a RegularExpressionsCore class which -contain a version of TPerlRegEx. However, direct use of the C PCRE library may -be desirable. - -The default makevp.bat, however, supplied with PCRE builds a version of PCRE -that is not usable with any version of C++ Builder because the compiler ships -with an embedded version of PCRE, version 2.01 from 1998! [See also the note -about BCC5.5 above.] If you want to use PCRE you'll need to rename the -functions (pcre_compile to pcre_compile_bcc, etc) or do as I have done and just -use the 16 bit versions. I'm using std::wstring everywhere anyway. Since the -embedded version of PCRE does not have the 16 bit function names, there is no -conflict. - -Building PCRE using a C++ Builder static library project file (recommended): - -1. Rename or remove pcre.h, pcreposi.h, and pcreposix.h from your C++ Builder -original include path. - -2. Download PCRE from pcre.org and extract to a directory. - -3. Rename pcre_chartables.c.dist to pcre_chartables.c, pcre.h.generic to -pcre.h, and config.h.generic to config.h. - -4. Edit pcre.h and pcre_config.c so that they include config.h. - -5. Edit config.h like so: - -Comment out the following lines: -#define PACKAGE "pcre" -#define PACKAGE_BUGREPORT "" -#define PACKAGE_NAME "PCRE" -#define PACKAGE_STRING "PCRE 8.32" -#define PACKAGE_TARNAME "pcre" -#define PACKAGE_URL "" -#define PACKAGE_VERSION "8.32" - -Add the following lines: -#ifndef SUPPORT_UTF -#define SUPPORT_UTF 100 // any value is fine -#endif - -#ifndef SUPPORT_UCP -#define SUPPORT_UCP 101 // any value is fine -#endif - -#ifndef SUPPORT_UCP -#define SUPPORT_PCRE16 102 // any value is fine -#endif - -#ifndef SUPPORT_UTF8 -#define SUPPORT_UTF8 103 // any value is fine -#endif - -6. Build a C++ Builder project using the IDE. Go to File / New / Other and -choose Static Library. You can name it pcre.cbproj or whatever. Now set your -paths by going to Project / Options. Set the Include path. Do this from the -"Base" option to apply to both Release and Debug builds. Now add the following -files to the project: - -pcre.h -pcre16_byte_order.c -pcre16_chartables.c -pcre16_compile.c -pcre16_config.c -pcre16_dfa_exec.c -pcre16_exec.c -pcre16_fullinfo.c -pcre16_get.c -pcre16_globals.c -pcre16_maketables.c -pcre16_newline.c -pcre16_ord2utf16.c -pcre16_printint.c -pcre16_refcount.c -pcre16_string_utils.c -pcre16_study.c -pcre16_tables.c -pcre16_ucd.c -pcre16_utf16_utils.c -pcre16_valid_utf16.c -pcre16_version.c -pcre16_xclass.c - -//Optional -pcre_version.c - -7. After compiling the .lib file, copy the .lib and header files to a project -you want to use PCRE with. Enjoy. - -Optional ... Building PCRE using the makevp.bat file: - -1. Edit makevp_c.txt and makevp_l.txt and change all the names to the 16 bit -versions. - -2. Edit makevp.bat and set the path to C++ Builder. Run makevp.bat. - -Another PCRE user added this comment: - -Another approach I successfully used for some years with BCB 5 and 6 was to -make sure that include and library paths of PCRE are configured before the -default paths of the IDE in the dialogs where one can manage those paths. -Afterwards one can open the project files using a text editor and manually add -the self created library for pcre itself, pcrecpp doesn't ship with the IDE, in -the library nodes where the IDE manages its own libraries to link against in -front of the IDE-own libraries. This way one can use the default PCRE function -names without getting access violations on runtime. - - <ALLLIB value="libpcre.lib $(LIBFILES) $(LIBRARIES) import32.lib cp32mt.lib"/> - - -BUILDING PCRE ON OPENVMS - -Stephen Hoffman sent the following, in December 2012: - -"Here <http://labs.hoffmanlabs.com/node/1847> is a very short write-up on the -OpenVMS port and here - -<http://labs.hoffmanlabs.com/labsnotes/pcre-vms-8_32.zip> - -is a zip with the OpenVMS files, and with one modified testing-related PCRE -file." This is a port of PCRE 8.32. - -Earlier, Dan Mooney sent the following comments about building PCRE on OpenVMS. -They relate to an older version of PCRE that used fewer source files, so the -exact commands will need changing. See the current list of source files above. - -"It was quite easy to compile and link the library. I don't have a formal -make file but the attached file [reproduced below] contains the OpenVMS DCL -commands I used to build the library. I had to add #define -POSIX_MALLOC_THRESHOLD 10 to pcre.h since it was not defined anywhere. - -The library was built on: -O/S: HP OpenVMS v7.3-1 -Compiler: Compaq C v6.5-001-48BCD -Linker: vA13-01 - -The test results did not match 100% due to the issues you mention in your -documentation regarding isprint(), iscntrl(), isgraph() and ispunct(). I -modified some of the character tables temporarily and was able to get the -results to match. Tests using the fr locale did not match since I don't have -that locale loaded. The study size was always reported to be 3 less than the -value in the standard test output files." - -========================= -$! This DCL procedure builds PCRE on OpenVMS -$! -$! I followed the instructions in the non-unix-use file in the distribution. -$! -$ COMPILE == "CC/LIST/NOMEMBER_ALIGNMENT/PREFIX_LIBRARY_ENTRIES=ALL_ENTRIES -$ COMPILE DFTABLES.C -$ LINK/EXE=DFTABLES.EXE DFTABLES.OBJ -$ RUN DFTABLES.EXE/OUTPUT=CHARTABLES.C -$ COMPILE MAKETABLES.C -$ COMPILE GET.C -$ COMPILE STUDY.C -$! I had to set POSIX_MALLOC_THRESHOLD to 10 in PCRE.H since the symbol -$! did not seem to be defined anywhere. -$! I edited pcre.h and added #DEFINE SUPPORT_UTF8 to enable UTF8 support. -$ COMPILE PCRE.C -$ LIB/CREATE PCRE MAKETABLES.OBJ, GET.OBJ, STUDY.OBJ, PCRE.OBJ -$! I had to set POSIX_MALLOC_THRESHOLD to 10 in PCRE.H since the symbol -$! did not seem to be defined anywhere. -$ COMPILE PCREPOSIX.C -$ LIB/CREATE PCREPOSIX PCREPOSIX.OBJ -$ COMPILE PCRETEST.C -$ LINK/EXE=PCRETEST.EXE PCRETEST.OBJ, PCRE/LIB, PCREPOSIX/LIB -$! C programs that want access to command line arguments must be -$! defined as a symbol -$ PCRETEST :== "$ SYS$ROADSUSERS:[DMOONEY.REGEXP]PCRETEST.EXE" -$! Arguments must be enclosed in quotes. -$ PCRETEST "-C" -$! Test results: -$! -$! The test results did not match 100%. The functions isprint(), iscntrl(), -$! isgraph() and ispunct() on OpenVMS must not produce the same results -$! as the system that built the test output files provided with the -$! distribution. -$! -$! The study size did not match and was always 3 less on OpenVMS. -$! -$! Locale could not be set to fr -$! -========================= - - -BUILDING PCRE ON STRATUS OPENVOS - -These notes on the port of PCRE to VOS (lightly edited) were supplied by -Ashutosh Warikoo, whose email address has the local part awarikoo and the -domain nse.co.in. The port was for version 7.9 in August 2009. - -1. Building PCRE - -I built pcre on OpenVOS Release 17.0.1at using GNU Tools 3.4a without any -problems. I used the following packages to build PCRE: - - ftp://ftp.stratus.com/pub/vos/posix/ga/posix.save.evf.gz - -Please read and follow the instructions that come with these packages. To start -the build of pcre, from the root of the package type: - - ./build.sh - -2. Installing PCRE - -Once you have successfully built PCRE, login to the SysAdmin group, switch to -the root user, and type - - [ !create_dir (master_disk)>usr --if needed ] - [ !create_dir (master_disk)>usr>local --if needed ] - !gmake install - -This installs PCRE and its man pages into /usr/local. You can add -(master_disk)>usr>local>bin to your command search paths, or if you are in -BASH, add /usr/local/bin to the PATH environment variable. - -4. Restrictions - -This port requires readline library optionally. However during the build I -faced some yet unexplored errors while linking with readline. As it was an -optional component I chose to disable it. - -5. Known Problems - -I ran the test suite, but you will have to be your own judge of whether this -command, and this port, suits your purposes. If you find any problems that -appear to be related to the port itself, please let me know. Please see the -build.log file in the root of the package also. - - -BUILDING PCRE ON NATIVE Z/OS AND Z/VM - -z/OS and z/VM are operating systems for mainframe computers, produced by IBM. -The character code used is EBCDIC, not ASCII or Unicode. In z/OS, UNIX APIs and -applications can be supported through UNIX System Services, and in such an -environment PCRE can be built in the same way as in other systems. However, in -native z/OS (without UNIX System Services) and in z/VM, special ports are -required. For details, please see this web site: - - http://www.zaconsultants.net - -There is also a mirror here: - - http://www.vsoft-software.com/downloads.html - -========================== -Last Updated: 14 May 2013 diff --git a/plugins/Pcre16/docs/doc/html/README.txt b/plugins/Pcre16/docs/doc/html/README.txt deleted file mode 100644 index 88f2dfd4ef..0000000000 --- a/plugins/Pcre16/docs/doc/html/README.txt +++ /dev/null @@ -1,991 +0,0 @@ -README file for PCRE (Perl-compatible regular expression library) ------------------------------------------------------------------ - -The latest release of PCRE is always available in three alternative formats -from: - - ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz - ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.bz2 - ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.zip - -There is a mailing list for discussion about the development of PCRE at -pcre-dev@exim.org. You can access the archives and subscribe or manage your -subscription here: - - https://lists.exim.org/mailman/listinfo/pcre-dev - -Please read the NEWS file if you are upgrading from a previous release. -The contents of this README file are: - - The PCRE APIs - Documentation for PCRE - Contributions by users of PCRE - Building PCRE on non-Unix-like systems - Building PCRE without using autotools - Building PCRE using autotools - Retrieving configuration information - Shared libraries - Cross-compiling using autotools - Using HP's ANSI C++ compiler (aCC) - Compiling in Tru64 using native compilers - Using Sun's compilers for Solaris - Using PCRE from MySQL - Making new tarballs - Testing PCRE - Character tables - File manifest - - -The PCRE APIs -------------- - -PCRE is written in C, and it has its own API. There are three sets of -functions, one for the 8-bit library, which processes strings of bytes, one for -the 16-bit library, which processes strings of 16-bit values, and one for the -32-bit library, which processes strings of 32-bit values. The distribution also -includes a set of C++ wrapper functions (see the pcrecpp man page for details), -courtesy of Google Inc., which can be used to call the 8-bit PCRE library from -C++. - -In addition, there is a set of C wrapper functions (again, just for the 8-bit -library) that are based on the POSIX regular expression API (see the pcreposix -man page). These end up in the library called libpcreposix. Note that this just -provides a POSIX calling interface to PCRE; the regular expressions themselves -still follow Perl syntax and semantics. The POSIX API is restricted, and does -not give full access to all of PCRE's facilities. - -The header file for the POSIX-style functions is called pcreposix.h. The -official POSIX name is regex.h, but I did not want to risk possible problems -with existing files of that name by distributing it that way. To use PCRE with -an existing program that uses the POSIX API, pcreposix.h will have to be -renamed or pointed at by a link. - -If you are using the POSIX interface to PCRE and there is already a POSIX regex -library installed on your system, as well as worrying about the regex.h header -file (as mentioned above), you must also take care when linking programs to -ensure that they link with PCRE's libpcreposix library. Otherwise they may pick -up the POSIX functions of the same name from the other library. - -One way of avoiding this confusion is to compile PCRE with the addition of --Dregcomp=PCREregcomp (and similarly for the other POSIX functions) to the -compiler flags (CFLAGS if you are using "configure" -- see below). This has the -effect of renaming the functions so that the names no longer clash. Of course, -you have to do the same thing for your applications, or write them using the -new names. - - -Documentation for PCRE ----------------------- - -If you install PCRE in the normal way on a Unix-like system, you will end up -with a set of man pages whose names all start with "pcre". The one that is just -called "pcre" lists all the others. In addition to these man pages, the PCRE -documentation is supplied in two other forms: - - 1. There are files called doc/pcre.txt, doc/pcregrep.txt, and - doc/pcretest.txt in the source distribution. The first of these is a - concatenation of the text forms of all the section 3 man pages except - the listing of pcredemo.c and those that summarize individual functions. - The other two are the text forms of the section 1 man pages for the - pcregrep and pcretest commands. These text forms are provided for ease of - scanning with text editors or similar tools. They are installed in - <prefix>/share/doc/pcre, where <prefix> is the installation prefix - (defaulting to /usr/local). - - 2. A set of files containing all the documentation in HTML form, hyperlinked - in various ways, and rooted in a file called index.html, is distributed in - doc/html and installed in <prefix>/share/doc/pcre/html. - -Users of PCRE have contributed files containing the documentation for various -releases in CHM format. These can be found in the Contrib directory of the FTP -site (see next section). - - -Contributions by users of PCRE ------------------------------- - -You can find contributions from PCRE users in the directory - - ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib - -There is a README file giving brief descriptions of what they are. Some are -complete in themselves; others are pointers to URLs containing relevant files. -Some of this material is likely to be well out-of-date. Several of the earlier -contributions provided support for compiling PCRE on various flavours of -Windows (I myself do not use Windows). Nowadays there is more Windows support -in the standard distribution, so these contibutions have been archived. - -A PCRE user maintains downloadable Windows binaries of the pcregrep and -pcretest programs here: - - http://www.rexegg.com/pcregrep-pcretest.html - - -Building PCRE on non-Unix-like systems --------------------------------------- - -For a non-Unix-like system, please read the comments in the file -NON-AUTOTOOLS-BUILD, though if your system supports the use of "configure" and -"make" you may be able to build PCRE using autotools in the same way as for -many Unix-like systems. - -PCRE can also be configured using the GUI facility provided by CMake's -cmake-gui command. This creates Makefiles, solution files, etc. The file -NON-AUTOTOOLS-BUILD has information about CMake. - -PCRE has been compiled on many different operating systems. It should be -straightforward to build PCRE on any system that has a Standard C compiler and -library, because it uses only Standard C functions. - - -Building PCRE without using autotools -------------------------------------- - -The use of autotools (in particular, libtool) is problematic in some -environments, even some that are Unix or Unix-like. See the NON-AUTOTOOLS-BUILD -file for ways of building PCRE without using autotools. - - -Building PCRE using autotools ------------------------------ - -If you are using HP's ANSI C++ compiler (aCC), please see the special note -in the section entitled "Using HP's ANSI C++ compiler (aCC)" below. - -The following instructions assume the use of the widely used "configure; make; -make install" (autotools) process. - -To build PCRE on system that supports autotools, first run the "configure" -command from the PCRE distribution directory, with your current directory set -to the directory where you want the files to be created. This command is a -standard GNU "autoconf" configuration script, for which generic instructions -are supplied in the file INSTALL. - -Most commonly, people build PCRE within its own distribution directory, and in -this case, on many systems, just running "./configure" is sufficient. However, -the usual methods of changing standard defaults are available. For example: - -CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local - -This command specifies that the C compiler should be run with the flags '-O2 --Wall' instead of the default, and that "make install" should install PCRE -under /opt/local instead of the default /usr/local. - -If you want to build in a different directory, just run "configure" with that -directory as current. For example, suppose you have unpacked the PCRE source -into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx: - -cd /build/pcre/pcre-xxx -/source/pcre/pcre-xxx/configure - -PCRE is written in C and is normally compiled as a C library. However, it is -possible to build it as a C++ library, though the provided building apparatus -does not have any features to support this. - -There are some optional features that can be included or omitted from the PCRE -library. They are also documented in the pcrebuild man page. - -. By default, both shared and static libraries are built. You can change this - by adding one of these options to the "configure" command: - - --disable-shared - --disable-static - - (See also "Shared libraries on Unix-like systems" below.) - -. By default, only the 8-bit library is built. If you add --enable-pcre16 to - the "configure" command, the 16-bit library is also built. If you add - --enable-pcre32 to the "configure" command, the 32-bit library is also built. - If you want only the 16-bit or 32-bit library, use --disable-pcre8 to disable - building the 8-bit library. - -. If you are building the 8-bit library and want to suppress the building of - the C++ wrapper library, you can add --disable-cpp to the "configure" - command. Otherwise, when "configure" is run without --disable-pcre8, it will - try to find a C++ compiler and C++ header files, and if it succeeds, it will - try to build the C++ wrapper. - -. If you want to include support for just-in-time compiling, which can give - large performance improvements on certain platforms, add --enable-jit to the - "configure" command. This support is available only for certain hardware - architectures. If you try to enable it on an unsupported architecture, there - will be a compile time error. - -. When JIT support is enabled, pcregrep automatically makes use of it, unless - you add --disable-pcregrep-jit to the "configure" command. - -. If you want to make use of the support for UTF-8 Unicode character strings in - the 8-bit library, or UTF-16 Unicode character strings in the 16-bit library, - or UTF-32 Unicode character strings in the 32-bit library, you must add - --enable-utf to the "configure" command. Without it, the code for handling - UTF-8, UTF-16 and UTF-8 is not included in the relevant library. Even - when --enable-utf is included, the use of a UTF encoding still has to be - enabled by an option at run time. When PCRE is compiled with this option, its - input can only either be ASCII or UTF-8/16/32, even when running on EBCDIC - platforms. It is not possible to use both --enable-utf and --enable-ebcdic at - the same time. - -. There are no separate options for enabling UTF-8, UTF-16 and UTF-32 - independently because that would allow ridiculous settings such as requesting - UTF-16 support while building only the 8-bit library. However, the option - --enable-utf8 is retained for backwards compatibility with earlier releases - that did not support 16-bit or 32-bit character strings. It is synonymous with - --enable-utf. It is not possible to configure one library with UTF support - and the other without in the same configuration. - -. If, in addition to support for UTF-8/16/32 character strings, you want to - include support for the \P, \p, and \X sequences that recognize Unicode - character properties, you must add --enable-unicode-properties to the - "configure" command. This adds about 30K to the size of the library (in the - form of a property table); only the basic two-letter properties such as Lu - are supported. - -. You can build PCRE to recognize either CR or LF or the sequence CRLF or any - of the preceding, or any of the Unicode newline sequences as indicating the - end of a line. Whatever you specify at build time is the default; the caller - of PCRE can change the selection at run time. The default newline indicator - is a single LF character (the Unix standard). You can specify the default - newline indicator by adding --enable-newline-is-cr or --enable-newline-is-lf - or --enable-newline-is-crlf or --enable-newline-is-anycrlf or - --enable-newline-is-any to the "configure" command, respectively. - - If you specify --enable-newline-is-cr or --enable-newline-is-crlf, some of - the standard tests will fail, because the lines in the test files end with - LF. Even if the files are edited to change the line endings, there are likely - to be some failures. With --enable-newline-is-anycrlf or - --enable-newline-is-any, many tests should succeed, but there may be some - failures. - -. By default, the sequence \R in a pattern matches any Unicode line ending - sequence. This is independent of the option specifying what PCRE considers to - be the end of a line (see above). However, the caller of PCRE can restrict \R - to match only CR, LF, or CRLF. You can make this the default by adding - --enable-bsr-anycrlf to the "configure" command (bsr = "backslash R"). - -. When called via the POSIX interface, PCRE uses malloc() to get additional - storage for processing capturing parentheses if there are more than 10 of - them in a pattern. You can increase this threshold by setting, for example, - - --with-posix-malloc-threshold=20 - - on the "configure" command. - -. PCRE has a counter that limits the depth of nesting of parentheses in a - pattern. This limits the amount of system stack that a pattern uses when it - is compiled. The default is 250, but you can change it by setting, for - example, - - --with-parens-nest-limit=500 - -. PCRE has a counter that can be set to limit the amount of resources it uses - when matching a pattern. If the limit is exceeded during a match, the match - fails. The default is ten million. You can change the default by setting, for - example, - - --with-match-limit=500000 - - on the "configure" command. This is just the default; individual calls to - pcre_exec() can supply their own value. There is more discussion on the - pcreapi man page. - -. There is a separate counter that limits the depth of recursive function calls - during a matching process. This also has a default of ten million, which is - essentially "unlimited". You can change the default by setting, for example, - - --with-match-limit-recursion=500000 - - Recursive function calls use up the runtime stack; running out of stack can - cause programs to crash in strange ways. There is a discussion about stack - sizes in the pcrestack man page. - -. The default maximum compiled pattern size is around 64K. You can increase - this by adding --with-link-size=3 to the "configure" command. In the 8-bit - library, PCRE then uses three bytes instead of two for offsets to different - parts of the compiled pattern. In the 16-bit library, --with-link-size=3 is - the same as --with-link-size=4, which (in both libraries) uses four-byte - offsets. Increasing the internal link size reduces performance. In the 32-bit - library, the only supported link size is 4. - -. You can build PCRE so that its internal match() function that is called from - pcre_exec() does not call itself recursively. Instead, it uses memory blocks - obtained from the heap via the special functions pcre_stack_malloc() and - pcre_stack_free() to save data that would otherwise be saved on the stack. To - build PCRE like this, use - - --disable-stack-for-recursion - - on the "configure" command. PCRE runs more slowly in this mode, but it may be - necessary in environments with limited stack sizes. This applies only to the - normal execution of the pcre_exec() function; if JIT support is being - successfully used, it is not relevant. Equally, it does not apply to - pcre_dfa_exec(), which does not use deeply nested recursion. There is a - discussion about stack sizes in the pcrestack man page. - -. For speed, PCRE uses four tables for manipulating and identifying characters - whose code point values are less than 256. By default, it uses a set of - tables for ASCII encoding that is part of the distribution. If you specify - - --enable-rebuild-chartables - - a program called dftables is compiled and run in the default C locale when - you obey "make". It builds a source file called pcre_chartables.c. If you do - not specify this option, pcre_chartables.c is created as a copy of - pcre_chartables.c.dist. See "Character tables" below for further information. - -. It is possible to compile PCRE for use on systems that use EBCDIC as their - character code (as opposed to ASCII/Unicode) by specifying - - --enable-ebcdic - - This automatically implies --enable-rebuild-chartables (see above). However, - when PCRE is built this way, it always operates in EBCDIC. It cannot support - both EBCDIC and UTF-8/16/32. There is a second option, --enable-ebcdic-nl25, - which specifies that the code value for the EBCDIC NL character is 0x25 - instead of the default 0x15. - -. In environments where valgrind is installed, if you specify - - --enable-valgrind - - PCRE will use valgrind annotations to mark certain memory regions as - unaddressable. This allows it to detect invalid memory accesses, and is - mostly useful for debugging PCRE itself. - -. In environments where the gcc compiler is used and lcov version 1.6 or above - is installed, if you specify - - --enable-coverage - - the build process implements a code coverage report for the test suite. The - report is generated by running "make coverage". If ccache is installed on - your system, it must be disabled when building PCRE for coverage reporting. - You can do this by setting the environment variable CCACHE_DISABLE=1 before - running "make" to build PCRE. There is more information about coverage - reporting in the "pcrebuild" documentation. - -. The pcregrep program currently supports only 8-bit data files, and so - requires the 8-bit PCRE library. It is possible to compile pcregrep to use - libz and/or libbz2, in order to read .gz and .bz2 files (respectively), by - specifying one or both of - - --enable-pcregrep-libz - --enable-pcregrep-libbz2 - - Of course, the relevant libraries must be installed on your system. - -. The default size (in bytes) of the internal buffer used by pcregrep can be - set by, for example: - - --with-pcregrep-bufsize=51200 - - The value must be a plain integer. The default is 20480. - -. It is possible to compile pcretest so that it links with the libreadline - or libedit libraries, by specifying, respectively, - - --enable-pcretest-libreadline or --enable-pcretest-libedit - - If this is done, when pcretest's input is from a terminal, it reads it using - the readline() function. This provides line-editing and history facilities. - Note that libreadline is GPL-licenced, so if you distribute a binary of - pcretest linked in this way, there may be licensing issues. These can be - avoided by linking with libedit (which has a BSD licence) instead. - - Enabling libreadline causes the -lreadline option to be added to the pcretest - build. In many operating environments with a sytem-installed readline - library this is sufficient. However, in some environments (e.g. if an - unmodified distribution version of readline is in use), it may be necessary - to specify something like LIBS="-lncurses" as well. This is because, to quote - the readline INSTALL, "Readline uses the termcap functions, but does not link - with the termcap or curses library itself, allowing applications which link - with readline the to choose an appropriate library." If you get error - messages about missing functions tgetstr, tgetent, tputs, tgetflag, or tgoto, - this is the problem, and linking with the ncurses library should fix it. - -The "configure" script builds the following files for the basic C library: - -. Makefile the makefile that builds the library -. config.h build-time configuration options for the library -. pcre.h the public PCRE header file -. pcre-config script that shows the building settings such as CFLAGS - that were set for "configure" -. libpcre.pc ) data for the pkg-config command -. libpcre16.pc ) -. libpcre32.pc ) -. libpcreposix.pc ) -. libtool script that builds shared and/or static libraries - -Versions of config.h and pcre.h are distributed in the PCRE tarballs under the -names config.h.generic and pcre.h.generic. These are provided for those who -have to built PCRE without using "configure" or CMake. If you use "configure" -or CMake, the .generic versions are not used. - -When building the 8-bit library, if a C++ compiler is found, the following -files are also built: - -. libpcrecpp.pc data for the pkg-config command -. pcrecpparg.h header file for calling PCRE via the C++ wrapper -. pcre_stringpiece.h header for the C++ "stringpiece" functions - -The "configure" script also creates config.status, which is an executable -script that can be run to recreate the configuration, and config.log, which -contains compiler output from tests that "configure" runs. - -Once "configure" has run, you can run "make". This builds the the libraries -libpcre, libpcre16 and/or libpcre32, and a test program called pcretest. If you -enabled JIT support with --enable-jit, a test program called pcre_jit_test is -built as well. - -If the 8-bit library is built, libpcreposix and the pcregrep command are also -built, and if a C++ compiler was found on your system, and you did not disable -it with --disable-cpp, "make" builds the C++ wrapper library, which is called -libpcrecpp, as well as some test programs called pcrecpp_unittest, -pcre_scanner_unittest, and pcre_stringpiece_unittest. - -The command "make check" runs all the appropriate tests. Details of the PCRE -tests are given below in a separate section of this document. - -You can use "make install" to install PCRE into live directories on your -system. The following are installed (file names are all relative to the -<prefix> that is set when "configure" is run): - - Commands (bin): - pcretest - pcregrep (if 8-bit support is enabled) - pcre-config - - Libraries (lib): - libpcre16 (if 16-bit support is enabled) - libpcre32 (if 32-bit support is enabled) - libpcre (if 8-bit support is enabled) - libpcreposix (if 8-bit support is enabled) - libpcrecpp (if 8-bit and C++ support is enabled) - - Configuration information (lib/pkgconfig): - libpcre16.pc - libpcre32.pc - libpcre.pc - libpcreposix.pc - libpcrecpp.pc (if C++ support is enabled) - - Header files (include): - pcre.h - pcreposix.h - pcre_scanner.h ) - pcre_stringpiece.h ) if C++ support is enabled - pcrecpp.h ) - pcrecpparg.h ) - - Man pages (share/man/man{1,3}): - pcregrep.1 - pcretest.1 - pcre-config.1 - pcre.3 - pcre*.3 (lots more pages, all starting "pcre") - - HTML documentation (share/doc/pcre/html): - index.html - *.html (lots more pages, hyperlinked from index.html) - - Text file documentation (share/doc/pcre): - AUTHORS - COPYING - ChangeLog - LICENCE - NEWS - README - pcre.txt (a concatenation of the man(3) pages) - pcretest.txt the pcretest man page - pcregrep.txt the pcregrep man page - pcre-config.txt the pcre-config man page - -If you want to remove PCRE from your system, you can run "make uninstall". -This removes all the files that "make install" installed. However, it does not -remove any directories, because these are often shared with other programs. - - -Retrieving configuration information ------------------------------------- - -Running "make install" installs the command pcre-config, which can be used to -recall information about the PCRE configuration and installation. For example: - - pcre-config --version - -prints the version number, and - - pcre-config --libs - -outputs information about where the library is installed. This command can be -included in makefiles for programs that use PCRE, saving the programmer from -having to remember too many details. - -The pkg-config command is another system for saving and retrieving information -about installed libraries. Instead of separate commands for each library, a -single command is used. For example: - - pkg-config --cflags pcre - -The data is held in *.pc files that are installed in a directory called -<prefix>/lib/pkgconfig. - - -Shared libraries ----------------- - -The default distribution builds PCRE as shared libraries and static libraries, -as long as the operating system supports shared libraries. Shared library -support relies on the "libtool" script which is built as part of the -"configure" process. - -The libtool script is used to compile and link both shared and static -libraries. They are placed in a subdirectory called .libs when they are newly -built. The programs pcretest and pcregrep are built to use these uninstalled -libraries (by means of wrapper scripts in the case of shared libraries). When -you use "make install" to install shared libraries, pcregrep and pcretest are -automatically re-built to use the newly installed shared libraries before being -installed themselves. However, the versions left in the build directory still -use the uninstalled libraries. - -To build PCRE using static libraries only you must use --disable-shared when -configuring it. For example: - -./configure --prefix=/usr/gnu --disable-shared - -Then run "make" in the usual way. Similarly, you can use --disable-static to -build only shared libraries. - - -Cross-compiling using autotools -------------------------------- - -You can specify CC and CFLAGS in the normal way to the "configure" command, in -order to cross-compile PCRE for some other host. However, you should NOT -specify --enable-rebuild-chartables, because if you do, the dftables.c source -file is compiled and run on the local host, in order to generate the inbuilt -character tables (the pcre_chartables.c file). This will probably not work, -because dftables.c needs to be compiled with the local compiler, not the cross -compiler. - -When --enable-rebuild-chartables is not specified, pcre_chartables.c is created -by making a copy of pcre_chartables.c.dist, which is a default set of tables -that assumes ASCII code. Cross-compiling with the default tables should not be -a problem. - -If you need to modify the character tables when cross-compiling, you should -move pcre_chartables.c.dist out of the way, then compile dftables.c by hand and -run it on the local host to make a new version of pcre_chartables.c.dist. -Then when you cross-compile PCRE this new version of the tables will be used. - - -Using HP's ANSI C++ compiler (aCC) ----------------------------------- - -Unless C++ support is disabled by specifying the "--disable-cpp" option of the -"configure" script, you must include the "-AA" option in the CXXFLAGS -environment variable in order for the C++ components to compile correctly. - -Also, note that the aCC compiler on PA-RISC platforms may have a defect whereby -needed libraries fail to get included when specifying the "-AA" compiler -option. If you experience unresolved symbols when linking the C++ programs, -use the workaround of specifying the following environment variable prior to -running the "configure" script: - - CXXLDFLAGS="-lstd_v2 -lCsup_v2" - - -Compiling in Tru64 using native compilers ------------------------------------------ - -The following error may occur when compiling with native compilers in the Tru64 -operating system: - - CXX libpcrecpp_la-pcrecpp.lo -cxx: Error: /usr/lib/cmplrs/cxx/V7.1-006/include/cxx/iosfwd, line 58: #error - directive: "cannot include iosfwd -- define __USE_STD_IOSTREAM to - override default - see section 7.1.2 of the C++ Using Guide" -#error "cannot include iosfwd -- define __USE_STD_IOSTREAM to override default -- see section 7.1.2 of the C++ Using Guide" - -This may be followed by other errors, complaining that 'namespace "std" has no -member'. The solution to this is to add the line - -#define __USE_STD_IOSTREAM 1 - -to the config.h file. - - -Using Sun's compilers for Solaris ---------------------------------- - -A user reports that the following configurations work on Solaris 9 sparcv9 and -Solaris 9 x86 (32-bit): - - Solaris 9 sparcv9: ./configure --disable-cpp CC=/bin/cc CFLAGS="-m64 -g" - Solaris 9 x86: ./configure --disable-cpp CC=/bin/cc CFLAGS="-g" - - -Using PCRE from MySQL ---------------------- - -On systems where both PCRE and MySQL are installed, it is possible to make use -of PCRE from within MySQL, as an alternative to the built-in pattern matching. -There is a web page that tells you how to do this: - - http://www.mysqludf.org/lib_mysqludf_preg/index.php - - -Making new tarballs -------------------- - -The command "make dist" creates three PCRE tarballs, in tar.gz, tar.bz2, and -zip formats. The command "make distcheck" does the same, but then does a trial -build of the new distribution to ensure that it works. - -If you have modified any of the man page sources in the doc directory, you -should first run the PrepareRelease script before making a distribution. This -script creates the .txt and HTML forms of the documentation from the man pages. - - -Testing PCRE ------------- - -To test the basic PCRE library on a Unix-like system, run the RunTest script. -There is another script called RunGrepTest that tests the options of the -pcregrep command. If the C++ wrapper library is built, three test programs -called pcrecpp_unittest, pcre_scanner_unittest, and pcre_stringpiece_unittest -are also built. When JIT support is enabled, another test program called -pcre_jit_test is built. - -Both the scripts and all the program tests are run if you obey "make check" or -"make test". For other environments, see the instructions in -NON-AUTOTOOLS-BUILD. - -The RunTest script runs the pcretest test program (which is documented in its -own man page) on each of the relevant testinput files in the testdata -directory, and compares the output with the contents of the corresponding -testoutput files. RunTest uses a file called testtry to hold the main output -from pcretest. Other files whose names begin with "test" are used as working -files in some tests. - -Some tests are relevant only when certain build-time options were selected. For -example, the tests for UTF-8/16/32 support are run only if --enable-utf was -used. RunTest outputs a comment when it skips a test. - -Many of the tests that are not skipped are run up to three times. The second -run forces pcre_study() to be called for all patterns except for a few in some -tests that are marked "never study" (see the pcretest program for how this is -done). If JIT support is available, the non-DFA tests are run a third time, -this time with a forced pcre_study() with the PCRE_STUDY_JIT_COMPILE option. -This testing can be suppressed by putting "nojit" on the RunTest command line. - -The entire set of tests is run once for each of the 8-bit, 16-bit and 32-bit -libraries that are enabled. If you want to run just one set of tests, call -RunTest with either the -8, -16 or -32 option. - -If valgrind is installed, you can run the tests under it by putting "valgrind" -on the RunTest command line. To run pcretest on just one or more specific test -files, give their numbers as arguments to RunTest, for example: - - RunTest 2 7 11 - -You can also specify ranges of tests such as 3-6 or 3- (meaning 3 to the -end), or a number preceded by ~ to exclude a test. For example: - - Runtest 3-15 ~10 - -This runs tests 3 to 15, excluding test 10, and just ~13 runs all the tests -except test 13. Whatever order the arguments are in, the tests are always run -in numerical order. - -You can also call RunTest with the single argument "list" to cause it to output -a list of tests. - -The first test file can be fed directly into the perltest.pl script to check -that Perl gives the same results. The only difference you should see is in the -first few lines, where the Perl version is given instead of the PCRE version. - -The second set of tests check pcre_fullinfo(), pcre_study(), -pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error -detection, and run-time flags that are specific to PCRE, as well as the POSIX -wrapper API. It also uses the debugging flags to check some of the internals of -pcre_compile(). - -If you build PCRE with a locale setting that is not the standard C locale, the -character tables may be different (see next paragraph). In some cases, this may -cause failures in the second set of tests. For example, in a locale where the -isprint() function yields TRUE for characters in the range 128-255, the use of -[:isascii:] inside a character class defines a different set of characters, and -this shows up in this test as a difference in the compiled code, which is being -listed for checking. Where the comparison test output contains [\x00-\x7f] the -test will contain [\x00-\xff], and similarly in some other cases. This is not a -bug in PCRE. - -The third set of tests checks pcre_maketables(), the facility for building a -set of character tables for a specific locale and using them instead of the -default tables. The tests make use of the "fr_FR" (French) locale. Before -running the test, the script checks for the presence of this locale by running -the "locale" command. If that command fails, or if it doesn't include "fr_FR" -in the list of available locales, the third test cannot be run, and a comment -is output to say why. If running this test produces instances of the error - - ** Failed to set locale "fr_FR" - -in the comparison output, it means that locale is not available on your system, -despite being listed by "locale". This does not mean that PCRE is broken. - -[If you are trying to run this test on Windows, you may be able to get it to -work by changing "fr_FR" to "french" everywhere it occurs. Alternatively, use -RunTest.bat. The version of RunTest.bat included with PCRE 7.4 and above uses -Windows versions of test 2. More info on using RunTest.bat is included in the -document entitled NON-UNIX-USE.] - -The fourth and fifth tests check the UTF-8/16/32 support and error handling and -internal UTF features of PCRE that are not relevant to Perl, respectively. The -sixth and seventh tests do the same for Unicode character properties support. - -The eighth, ninth, and tenth tests check the pcre_dfa_exec() alternative -matching function, in non-UTF-8/16/32 mode, UTF-8/16/32 mode, and UTF-8/16/32 -mode with Unicode property support, respectively. - -The eleventh test checks some internal offsets and code size features; it is -run only when the default "link size" of 2 is set (in other cases the sizes -change) and when Unicode property support is enabled. - -The twelfth test is run only when JIT support is available, and the thirteenth -test is run only when JIT support is not available. They test some JIT-specific -features such as information output from pcretest about JIT compilation. - -The fourteenth, fifteenth, and sixteenth tests are run only in 8-bit mode, and -the seventeenth, eighteenth, and nineteenth tests are run only in 16/32-bit -mode. These are tests that generate different output in the two modes. They are -for general cases, UTF-8/16/32 support, and Unicode property support, -respectively. - -The twentieth test is run only in 16/32-bit mode. It tests some specific -16/32-bit features of the DFA matching engine. - -The twenty-first and twenty-second tests are run only in 16/32-bit mode, when -the link size is set to 2 for the 16-bit library. They test reloading -pre-compiled patterns. - -The twenty-third and twenty-fourth tests are run only in 16-bit mode. They are -for general cases, and UTF-16 support, respectively. - -The twenty-fifth and twenty-sixth tests are run only in 32-bit mode. They are -for general cases, and UTF-32 support, respectively. - - -Character tables ----------------- - -For speed, PCRE uses four tables for manipulating and identifying characters -whose code point values are less than 256. The final argument of the -pcre_compile() function is a pointer to a block of memory containing the -concatenated tables. A call to pcre_maketables() can be used to generate a set -of tables in the current locale. If the final argument for pcre_compile() is -passed as NULL, a set of default tables that is built into the binary is used. - -The source file called pcre_chartables.c contains the default set of tables. By -default, this is created as a copy of pcre_chartables.c.dist, which contains -tables for ASCII coding. However, if --enable-rebuild-chartables is specified -for ./configure, a different version of pcre_chartables.c is built by the -program dftables (compiled from dftables.c), which uses the ANSI C character -handling functions such as isalnum(), isalpha(), isupper(), islower(), etc. to -build the table sources. This means that the default C locale which is set for -your system will control the contents of these default tables. You can change -the default tables by editing pcre_chartables.c and then re-building PCRE. If -you do this, you should take care to ensure that the file does not get -automatically re-generated. The best way to do this is to move -pcre_chartables.c.dist out of the way and replace it with your customized -tables. - -When the dftables program is run as a result of --enable-rebuild-chartables, -it uses the default C locale that is set on your system. It does not pay -attention to the LC_xxx environment variables. In other words, it uses the -system's default locale rather than whatever the compiling user happens to have -set. If you really do want to build a source set of character tables in a -locale that is specified by the LC_xxx variables, you can run the dftables -program by hand with the -L option. For example: - - ./dftables -L pcre_chartables.c.special - -The first two 256-byte tables provide lower casing and case flipping functions, -respectively. The next table consists of three 32-byte bit maps which identify -digits, "word" characters, and white space, respectively. These are used when -building 32-byte bit maps that represent character classes for code points less -than 256. - -The final 256-byte table has bits indicating various character types, as -follows: - - 1 white space character - 2 letter - 4 decimal digit - 8 hexadecimal digit - 16 alphanumeric or '_' - 128 regular expression metacharacter or binary zero - -You should not alter the set of characters that contain the 128 bit, as that -will cause PCRE to malfunction. - - -File manifest -------------- - -The distribution should contain the files listed below. Where a file name is -given as pcre[16|32]_xxx it means that there are three files, one with the name -pcre_xxx, one with the name pcre16_xx, and a third with the name pcre32_xxx. - -(A) Source files of the PCRE library functions and their headers: - - dftables.c auxiliary program for building pcre_chartables.c - when --enable-rebuild-chartables is specified - - pcre_chartables.c.dist a default set of character tables that assume ASCII - coding; used, unless --enable-rebuild-chartables is - specified, by copying to pcre[16]_chartables.c - - pcreposix.c ) - pcre[16|32]_byte_order.c ) - pcre[16|32]_compile.c ) - pcre[16|32]_config.c ) - pcre[16|32]_dfa_exec.c ) - pcre[16|32]_exec.c ) - pcre[16|32]_fullinfo.c ) - pcre[16|32]_get.c ) sources for the functions in the library, - pcre[16|32]_globals.c ) and some internal functions that they use - pcre[16|32]_jit_compile.c ) - pcre[16|32]_maketables.c ) - pcre[16|32]_newline.c ) - pcre[16|32]_refcount.c ) - pcre[16|32]_string_utils.c ) - pcre[16|32]_study.c ) - pcre[16|32]_tables.c ) - pcre[16|32]_ucd.c ) - pcre[16|32]_version.c ) - pcre[16|32]_xclass.c ) - pcre_ord2utf8.c ) - pcre_valid_utf8.c ) - pcre16_ord2utf16.c ) - pcre16_utf16_utils.c ) - pcre16_valid_utf16.c ) - pcre32_utf32_utils.c ) - pcre32_valid_utf32.c ) - - pcre[16|32]_printint.c ) debugging function that is used by pcretest, - ) and can also be #included in pcre_compile() - - pcre.h.in template for pcre.h when built by "configure" - pcreposix.h header for the external POSIX wrapper API - pcre_internal.h header for internal use - sljit/* 16 files that make up the JIT compiler - ucp.h header for Unicode property handling - - config.h.in template for config.h, which is built by "configure" - - pcrecpp.h public header file for the C++ wrapper - pcrecpparg.h.in template for another C++ header file - pcre_scanner.h public header file for C++ scanner functions - pcrecpp.cc ) - pcre_scanner.cc ) source for the C++ wrapper library - - pcre_stringpiece.h.in template for pcre_stringpiece.h, the header for the - C++ stringpiece functions - pcre_stringpiece.cc source for the C++ stringpiece functions - -(B) Source files for programs that use PCRE: - - pcredemo.c simple demonstration of coding calls to PCRE - pcregrep.c source of a grep utility that uses PCRE - pcretest.c comprehensive test program - -(C) Auxiliary files: - - 132html script to turn "man" pages into HTML - AUTHORS information about the author of PCRE - ChangeLog log of changes to the code - CleanTxt script to clean nroff output for txt man pages - Detrail script to remove trailing spaces - HACKING some notes about the internals of PCRE - INSTALL generic installation instructions - LICENCE conditions for the use of PCRE - COPYING the same, using GNU's standard name - Makefile.in ) template for Unix Makefile, which is built by - ) "configure" - Makefile.am ) the automake input that was used to create - ) Makefile.in - NEWS important changes in this release - NON-UNIX-USE the previous name for NON-AUTOTOOLS-BUILD - NON-AUTOTOOLS-BUILD notes on building PCRE without using autotools - PrepareRelease script to make preparations for "make dist" - README this file - RunTest a Unix shell script for running tests - RunGrepTest a Unix shell script for pcregrep tests - aclocal.m4 m4 macros (generated by "aclocal") - config.guess ) files used by libtool, - config.sub ) used only when building a shared library - configure a configuring shell script (built by autoconf) - configure.ac ) the autoconf input that was used to build - ) "configure" and config.h - depcomp ) script to find program dependencies, generated by - ) automake - doc/*.3 man page sources for PCRE - doc/*.1 man page sources for pcregrep and pcretest - doc/index.html.src the base HTML page - doc/html/* HTML documentation - doc/pcre.txt plain text version of the man pages - doc/pcretest.txt plain text documentation of test program - doc/perltest.txt plain text documentation of Perl test program - install-sh a shell script for installing files - libpcre16.pc.in template for libpcre16.pc for pkg-config - libpcre32.pc.in template for libpcre32.pc for pkg-config - libpcre.pc.in template for libpcre.pc for pkg-config - libpcreposix.pc.in template for libpcreposix.pc for pkg-config - libpcrecpp.pc.in template for libpcrecpp.pc for pkg-config - ltmain.sh file used to build a libtool script - missing ) common stub for a few missing GNU programs while - ) installing, generated by automake - mkinstalldirs script for making install directories - perltest.pl Perl test program - pcre-config.in source of script which retains PCRE information - pcre_jit_test.c test program for the JIT compiler - pcrecpp_unittest.cc ) - pcre_scanner_unittest.cc ) test programs for the C++ wrapper - pcre_stringpiece_unittest.cc ) - testdata/testinput* test data for main library tests - testdata/testoutput* expected test results - testdata/grep* input and output for pcregrep tests - testdata/* other supporting test files - -(D) Auxiliary files for cmake support - - cmake/COPYING-CMAKE-SCRIPTS - cmake/FindPackageHandleStandardArgs.cmake - cmake/FindEditline.cmake - cmake/FindReadline.cmake - CMakeLists.txt - config-cmake.h.in - -(E) Auxiliary files for VPASCAL - - makevp.bat - makevp_c.txt - makevp_l.txt - pcregexp.pas - -(F) Auxiliary files for building PCRE "by hand" - - pcre.h.generic ) a version of the public PCRE header file - ) for use in non-"configure" environments - config.h.generic ) a version of config.h for use in non-"configure" - ) environments - -(F) Miscellaneous - - RunTest.bat a script for running tests under Windows - -Philip Hazel -Email local part: ph10 -Email domain: cam.ac.uk -Last updated: 17 January 2014 diff --git a/plugins/Pcre16/docs/doc/html/index.html b/plugins/Pcre16/docs/doc/html/index.html deleted file mode 100644 index 352c55df2f..0000000000 --- a/plugins/Pcre16/docs/doc/html/index.html +++ /dev/null @@ -1,185 +0,0 @@ -<html> -<!-- This is a manually maintained file that is the root of the HTML version of - the PCRE documentation. When the HTML documents are built from the man - page versions, the entire doc/html directory is emptied, this file is then - copied into doc/html/index.html, and the remaining files therein are - created by the 132html script. ---> -<head> -<title>PCRE specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>Perl-compatible Regular Expressions (PCRE)</h1> -<p> -The HTML documentation for PCRE consists of a number of pages that are listed -below in alphabetical order. If you are new to PCRE, please read the first one -first. -</p> - -<table> -<tr><td><a href="pcre.html">pcre</a></td> - <td> Introductory page</td></tr> - -<tr><td><a href="pcre-config.html">pcre-config</a></td> - <td> Information about the installation configuration</td></tr> - -<tr><td><a href="pcre16.html">pcre16</a></td> - <td> Discussion of the 16-bit PCRE library</td></tr> - -<tr><td><a href="pcre32.html">pcre32</a></td> - <td> Discussion of the 32-bit PCRE library</td></tr> - -<tr><td><a href="pcreapi.html">pcreapi</a></td> - <td> PCRE's native API</td></tr> - -<tr><td><a href="pcrebuild.html">pcrebuild</a></td> - <td> Building PCRE</td></tr> - -<tr><td><a href="pcrecallout.html">pcrecallout</a></td> - <td> The <i>callout</i> facility</td></tr> - -<tr><td><a href="pcrecompat.html">pcrecompat</a></td> - <td> Compability with Perl</td></tr> - -<tr><td><a href="pcrecpp.html">pcrecpp</a></td> - <td> The C++ wrapper for the PCRE library</td></tr> - -<tr><td><a href="pcredemo.html">pcredemo</a></td> - <td> A demonstration C program that uses the PCRE library</td></tr> - -<tr><td><a href="pcregrep.html">pcregrep</a></td> - <td> The <b>pcregrep</b> command</td></tr> - -<tr><td><a href="pcrejit.html">pcrejit</a></td> - <td> Discussion of the just-in-time optimization support</td></tr> - -<tr><td><a href="pcrelimits.html">pcrelimits</a></td> - <td> Details of size and other limits</td></tr> - -<tr><td><a href="pcrematching.html">pcrematching</a></td> - <td> Discussion of the two matching algorithms</td></tr> - -<tr><td><a href="pcrepartial.html">pcrepartial</a></td> - <td> Using PCRE for partial matching</td></tr> - -<tr><td><a href="pcrepattern.html">pcrepattern</a></td> - <td> Specification of the regular expressions supported by PCRE</td></tr> - -<tr><td><a href="pcreperform.html">pcreperform</a></td> - <td> Some comments on performance</td></tr> - -<tr><td><a href="pcreposix.html">pcreposix</a></td> - <td> The POSIX API to the PCRE 8-bit library</td></tr> - -<tr><td><a href="pcreprecompile.html">pcreprecompile</a></td> - <td> How to save and re-use compiled patterns</td></tr> - -<tr><td><a href="pcresample.html">pcresample</a></td> - <td> Discussion of the pcredemo program</td></tr> - -<tr><td><a href="pcrestack.html">pcrestack</a></td> - <td> Discussion of PCRE's stack usage</td></tr> - -<tr><td><a href="pcresyntax.html">pcresyntax</a></td> - <td> Syntax quick-reference summary</td></tr> - -<tr><td><a href="pcretest.html">pcretest</a></td> - <td> The <b>pcretest</b> command for testing PCRE</td></tr> - -<tr><td><a href="pcreunicode.html">pcreunicode</a></td> - <td> Discussion of Unicode and UTF-8/UTF-16/UTF-32 support</td></tr> -</table> - -<p> -There are also individual pages that summarize the interface for each function -in the library. There is a single page for each triple of 8-bit/16-bit/32-bit -functions. -</p> - -<table> - -<tr><td><a href="pcre_assign_jit_stack.html">pcre_assign_jit_stack</a></td> - <td> Assign stack for JIT matching</td></tr> - -<tr><td><a href="pcre_compile.html">pcre_compile</a></td> - <td> Compile a regular expression</td></tr> - -<tr><td><a href="pcre_compile2.html">pcre_compile2</a></td> - <td> Compile a regular expression (alternate interface)</td></tr> - -<tr><td><a href="pcre_config.html">pcre_config</a></td> - <td> Show build-time configuration options</td></tr> - -<tr><td><a href="pcre_copy_named_substring.html">pcre_copy_named_substring</a></td> - <td> Extract named substring into given buffer</td></tr> - -<tr><td><a href="pcre_copy_substring.html">pcre_copy_substring</a></td> - <td> Extract numbered substring into given buffer</td></tr> - -<tr><td><a href="pcre_dfa_exec.html">pcre_dfa_exec</a></td> - <td> Match a compiled pattern to a subject string - (DFA algorithm; <i>not</i> Perl compatible)</td></tr> - -<tr><td><a href="pcre_exec.html">pcre_exec</a></td> - <td> Match a compiled pattern to a subject string - (Perl compatible)</td></tr> - -<tr><td><a href="pcre_free_study.html">pcre_free_study</a></td> - <td> Free study data</td></tr> - -<tr><td><a href="pcre_free_substring.html">pcre_free_substring</a></td> - <td> Free extracted substring</td></tr> - -<tr><td><a href="pcre_free_substring_list.html">pcre_free_substring_list</a></td> - <td> Free list of extracted substrings</td></tr> - -<tr><td><a href="pcre_fullinfo.html">pcre_fullinfo</a></td> - <td> Extract information about a pattern</td></tr> - -<tr><td><a href="pcre_get_named_substring.html">pcre_get_named_substring</a></td> - <td> Extract named substring into new memory</td></tr> - -<tr><td><a href="pcre_get_stringnumber.html">pcre_get_stringnumber</a></td> - <td> Convert captured string name to number</td></tr> - -<tr><td><a href="pcre_get_stringtable_entries.html">pcre_get_stringtable_entries</a></td> - <td> Find table entries for given string name</td></tr> - -<tr><td><a href="pcre_get_substring.html">pcre_get_substring</a></td> - <td> Extract numbered substring into new memory</td></tr> - -<tr><td><a href="pcre_get_substring_list.html">pcre_get_substring_list</a></td> - <td> Extract all substrings into new memory</td></tr> - -<tr><td><a href="pcre_jit_exec.html">pcre_jit_exec</a></td> - <td> Fast path interface to JIT matching</td></tr> - -<tr><td><a href="pcre_jit_stack_alloc.html">pcre_jit_stack_alloc</a></td> - <td> Create a stack for JIT matching</td></tr> - -<tr><td><a href="pcre_jit_stack_free.html">pcre_jit_stack_free</a></td> - <td> Free a JIT matching stack</td></tr> - -<tr><td><a href="pcre_maketables.html">pcre_maketables</a></td> - <td> Build character tables in current locale</td></tr> - -<tr><td><a href="pcre_pattern_to_host_byte_order.html">pcre_pattern_to_host_byte_order</a></td> - <td> Convert compiled pattern to host byte order if necessary</td></tr> - -<tr><td><a href="pcre_refcount.html">pcre_refcount</a></td> - <td> Maintain reference count in compiled pattern</td></tr> - -<tr><td><a href="pcre_study.html">pcre_study</a></td> - <td> Study a compiled pattern</td></tr> - -<tr><td><a href="pcre_utf16_to_host_byte_order.html">pcre_utf16_to_host_byte_order</a></td> - <td> Convert UTF-16 string to host byte order if necessary</td></tr> - -<tr><td><a href="pcre_utf32_to_host_byte_order.html">pcre_utf32_to_host_byte_order</a></td> - <td> Convert UTF-32 string to host byte order if necessary</td></tr> - -<tr><td><a href="pcre_version.html">pcre_version</a></td> - <td> Return PCRE version and release date</td></tr> -</table> - -</html> diff --git a/plugins/Pcre16/docs/doc/html/pcre-config.html b/plugins/Pcre16/docs/doc/html/pcre-config.html deleted file mode 100644 index 56a8060492..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre-config.html +++ /dev/null @@ -1,109 +0,0 @@ -<html> -<head> -<title>pcre-config specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre-config man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">SYNOPSIS</a> -<li><a name="TOC2" href="#SEC2">DESCRIPTION</a> -<li><a name="TOC3" href="#SEC3">OPTIONS</a> -<li><a name="TOC4" href="#SEC4">SEE ALSO</a> -<li><a name="TOC5" href="#SEC5">AUTHOR</a> -<li><a name="TOC6" href="#SEC6">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">SYNOPSIS</a><br> -<P> -<b>pcre-config [--prefix] [--exec-prefix] [--version] [--libs]</b> -<b> [--libs16] [--libs32] [--libs-cpp] [--libs-posix]</b> -<b> [--cflags] [--cflags-posix]</b> -</P> -<br><a name="SEC2" href="#TOC1">DESCRIPTION</a><br> -<P> -<b>pcre-config</b> returns the configuration of the installed PCRE -libraries and the options required to compile a program to use them. Some of -the options apply only to the 8-bit, or 16-bit, or 32-bit libraries, -respectively, and are -not available if only one of those libraries has been built. If an unavailable -option is encountered, the "usage" information is output. -</P> -<br><a name="SEC3" href="#TOC1">OPTIONS</a><br> -<P> -<b>--prefix</b> -Writes the directory prefix used in the PCRE installation for architecture -independent files (<i>/usr</i> on many systems, <i>/usr/local</i> on some -systems) to the standard output. -</P> -<P> -<b>--exec-prefix</b> -Writes the directory prefix used in the PCRE installation for architecture -dependent files (normally the same as <b>--prefix</b>) to the standard output. -</P> -<P> -<b>--version</b> -Writes the version number of the installed PCRE libraries to the standard -output. -</P> -<P> -<b>--libs</b> -Writes to the standard output the command line options required to link -with the 8-bit PCRE library (<b>-lpcre</b> on many systems). -</P> -<P> -<b>--libs16</b> -Writes to the standard output the command line options required to link -with the 16-bit PCRE library (<b>-lpcre16</b> on many systems). -</P> -<P> -<b>--libs32</b> -Writes to the standard output the command line options required to link -with the 32-bit PCRE library (<b>-lpcre32</b> on many systems). -</P> -<P> -<b>--libs-cpp</b> -Writes to the standard output the command line options required to link with -PCRE's C++ wrapper library (<b>-lpcrecpp</b> <b>-lpcre</b> on many -systems). -</P> -<P> -<b>--libs-posix</b> -Writes to the standard output the command line options required to link with -PCRE's POSIX API wrapper library (<b>-lpcreposix</b> <b>-lpcre</b> on many -systems). -</P> -<P> -<b>--cflags</b> -Writes to the standard output the command line options required to compile -files that use PCRE (this may include some <b>-I</b> options, but is blank on -many systems). -</P> -<P> -<b>--cflags-posix</b> -Writes to the standard output the command line options required to compile -files that use PCRE's POSIX API wrapper library (this may include some <b>-I</b> -options, but is blank on many systems). -</P> -<br><a name="SEC4" href="#TOC1">SEE ALSO</a><br> -<P> -<b>pcre(3)</b> -</P> -<br><a name="SEC5" href="#TOC1">AUTHOR</a><br> -<P> -This manual page was originally written by Mark Baker for the Debian GNU/Linux -system. It has been subsequently revised as a generic PCRE man page. -</P> -<br><a name="SEC6" href="#TOC1">REVISION</a><br> -<P> -Last updated: 24 June 2012 -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre.html b/plugins/Pcre16/docs/doc/html/pcre.html deleted file mode 100644 index c2b29aa812..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre.html +++ /dev/null @@ -1,213 +0,0 @@ -<html> -<head> -<title>pcre specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">INTRODUCTION</a> -<li><a name="TOC2" href="#SEC2">SECURITY CONSIDERATIONS</a> -<li><a name="TOC3" href="#SEC3">USER DOCUMENTATION</a> -<li><a name="TOC4" href="#SEC4">AUTHOR</a> -<li><a name="TOC5" href="#SEC5">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">INTRODUCTION</a><br> -<P> -The PCRE library is a set of functions that implement regular expression -pattern matching using the same syntax and semantics as Perl, with just a few -differences. Some features that appeared in Python and PCRE before they -appeared in Perl are also available using the Python syntax, there is some -support for one or two .NET and Oniguruma syntax items, and there is an option -for requesting some minor changes that give better JavaScript compatibility. -</P> -<P> -Starting with release 8.30, it is possible to compile two separate PCRE -libraries: the original, which supports 8-bit character strings (including -UTF-8 strings), and a second library that supports 16-bit character strings -(including UTF-16 strings). The build process allows either one or both to be -built. The majority of the work to make this possible was done by Zoltan -Herczeg. -</P> -<P> -Starting with release 8.32 it is possible to compile a third separate PCRE -library that supports 32-bit character strings (including UTF-32 strings). The -build process allows any combination of the 8-, 16- and 32-bit libraries. The -work to make this possible was done by Christian Persch. -</P> -<P> -The three libraries contain identical sets of functions, except that the names -in the 16-bit library start with <b>pcre16_</b> instead of <b>pcre_</b>, and the -names in the 32-bit library start with <b>pcre32_</b> instead of <b>pcre_</b>. To -avoid over-complication and reduce the documentation maintenance load, most of -the documentation describes the 8-bit library, with the differences for the -16-bit and 32-bit libraries described separately in the -<a href="pcre16.html"><b>pcre16</b></a> -and -<a href="pcre32.html"><b>pcre32</b></a> -pages. References to functions or structures of the form <i>pcre[16|32]_xxx</i> -should be read as meaning "<i>pcre_xxx</i> when using the 8-bit library, -<i>pcre16_xxx</i> when using the 16-bit library, or <i>pcre32_xxx</i> when using -the 32-bit library". -</P> -<P> -The current implementation of PCRE corresponds approximately with Perl 5.12, -including support for UTF-8/16/32 encoded strings and Unicode general category -properties. However, UTF-8/16/32 and Unicode support has to be explicitly -enabled; it is not the default. The Unicode tables correspond to Unicode -release 6.3.0. -</P> -<P> -In addition to the Perl-compatible matching function, PCRE contains an -alternative function that matches the same compiled patterns in a different -way. In certain circumstances, the alternative function has some advantages. -For a discussion of the two matching algorithms, see the -<a href="pcrematching.html"><b>pcrematching</b></a> -page. -</P> -<P> -PCRE is written in C and released as a C library. A number of people have -written wrappers and interfaces of various kinds. In particular, Google Inc. -have provided a comprehensive C++ wrapper for the 8-bit library. This is now -included as part of the PCRE distribution. The -<a href="pcrecpp.html"><b>pcrecpp</b></a> -page has details of this interface. Other people's contributions can be found -in the <i>Contrib</i> directory at the primary FTP site, which is: -<a href="ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre">ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre</a> -</P> -<P> -Details of exactly which Perl regular expression features are and are not -supported by PCRE are given in separate documents. See the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -and -<a href="pcrecompat.html"><b>pcrecompat</b></a> -pages. There is a syntax summary in the -<a href="pcresyntax.html"><b>pcresyntax</b></a> -page. -</P> -<P> -Some features of PCRE can be included, excluded, or changed when the library is -built. The -<a href="pcre_config.html"><b>pcre_config()</b></a> -function makes it possible for a client to discover which features are -available. The features themselves are described in the -<a href="pcrebuild.html"><b>pcrebuild</b></a> -page. Documentation about building PCRE for various operating systems can be -found in the -<a href="README.txt"><b>README</b></a> -and -<a href="NON-AUTOTOOLS-BUILD.txt"><b>NON-AUTOTOOLS_BUILD</b></a> -files in the source distribution. -</P> -<P> -The libraries contains a number of undocumented internal functions and data -tables that are used by more than one of the exported external functions, but -which are not intended for use by external callers. Their names all begin with -"_pcre_" or "_pcre16_" or "_pcre32_", which hopefully will not provoke any name -clashes. In some environments, it is possible to control which external symbols -are exported when a shared library is built, and in these cases the -undocumented symbols are not exported. -</P> -<br><a name="SEC2" href="#TOC1">SECURITY CONSIDERATIONS</a><br> -<P> -If you are using PCRE in a non-UTF application that permits users to supply -arbitrary patterns for compilation, you should be aware of a feature that -allows users to turn on UTF support from within a pattern, provided that PCRE -was built with UTF support. For example, an 8-bit pattern that begins with -"(*UTF8)" or "(*UTF)" turns on UTF-8 mode, which interprets patterns and -subjects as strings of UTF-8 characters instead of individual 8-bit characters. -This causes both the pattern and any data against which it is matched to be -checked for UTF-8 validity. If the data string is very long, such a check might -use sufficiently many resources as to cause your application to lose -performance. -</P> -<P> -One way of guarding against this possibility is to use the -<b>pcre_fullinfo()</b> function to check the compiled pattern's options for UTF. -Alternatively, from release 8.33, you can set the PCRE_NEVER_UTF option at -compile time. This causes an compile time error if a pattern contains a -UTF-setting sequence. -</P> -<P> -If your application is one that supports UTF, be aware that validity checking -can take time. If the same data string is to be matched many times, you can use -the PCRE_NO_UTF[8|16|32]_CHECK option for the second and subsequent matches to -save redundant checks. -</P> -<P> -Another way that performance can be hit is by running a pattern that has a very -large search tree against a string that will never match. Nested unlimited -repeats in a pattern are a common example. PCRE provides some protection -against this: see the PCRE_EXTRA_MATCH_LIMIT feature in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page. -</P> -<br><a name="SEC3" href="#TOC1">USER DOCUMENTATION</a><br> -<P> -The user documentation for PCRE comprises a number of different sections. In -the "man" format, each of these is a separate "man page". In the HTML format, -each is a separate page, linked from the index page. In the plain text format, -the descriptions of the <b>pcregrep</b> and <b>pcretest</b> programs are in files -called <b>pcregrep.txt</b> and <b>pcretest.txt</b>, respectively. The remaining -sections, except for the <b>pcredemo</b> section (which is a program listing), -are concatenated in <b>pcre.txt</b>, for ease of searching. The sections are as -follows: -<pre> - pcre this document - pcre-config show PCRE installation configuration information - pcre16 details of the 16-bit library - pcre32 details of the 32-bit library - pcreapi details of PCRE's native C API - pcrebuild building PCRE - pcrecallout details of the callout feature - pcrecompat discussion of Perl compatibility - pcrecpp details of the C++ wrapper for the 8-bit library - pcredemo a demonstration C program that uses PCRE - pcregrep description of the <b>pcregrep</b> command (8-bit only) - pcrejit discussion of the just-in-time optimization support - pcrelimits details of size and other limits - pcrematching discussion of the two matching algorithms - pcrepartial details of the partial matching facility - pcrepattern syntax and semantics of supported regular expressions - pcreperform discussion of performance issues - pcreposix the POSIX-compatible C API for the 8-bit library - pcreprecompile details of saving and re-using precompiled patterns - pcresample discussion of the pcredemo program - pcrestack discussion of stack usage - pcresyntax quick syntax reference - pcretest description of the <b>pcretest</b> testing command - pcreunicode discussion of Unicode and UTF-8/16/32 support -</pre> -In the "man" and HTML formats, there is also a short page for each C library -function, listing its arguments and results. -</P> -<br><a name="SEC4" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<P> -Putting an actual email address here seems to have been a spam magnet, so I've -taken it away. If you want to email me, use my two initials, followed by the -two digits 10, at the domain cam.ac.uk. -</P> -<br><a name="SEC5" href="#TOC1">REVISION</a><br> -<P> -Last updated: 08 January 2014 -<br> -Copyright © 1997-2014 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre16.html b/plugins/Pcre16/docs/doc/html/pcre16.html deleted file mode 100644 index f00859f052..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre16.html +++ /dev/null @@ -1,384 +0,0 @@ -<html> -<head> -<title>pcre16 specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre16 man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">PCRE 16-BIT API BASIC FUNCTIONS</a> -<li><a name="TOC2" href="#SEC2">PCRE 16-BIT API STRING EXTRACTION FUNCTIONS</a> -<li><a name="TOC3" href="#SEC3">PCRE 16-BIT API AUXILIARY FUNCTIONS</a> -<li><a name="TOC4" href="#SEC4">PCRE 16-BIT API INDIRECTED FUNCTIONS</a> -<li><a name="TOC5" href="#SEC5">PCRE 16-BIT API 16-BIT-ONLY FUNCTION</a> -<li><a name="TOC6" href="#SEC6">THE PCRE 16-BIT LIBRARY</a> -<li><a name="TOC7" href="#SEC7">THE HEADER FILE</a> -<li><a name="TOC8" href="#SEC8">THE LIBRARY NAME</a> -<li><a name="TOC9" href="#SEC9">STRING TYPES</a> -<li><a name="TOC10" href="#SEC10">STRUCTURE TYPES</a> -<li><a name="TOC11" href="#SEC11">16-BIT FUNCTIONS</a> -<li><a name="TOC12" href="#SEC12">SUBJECT STRING OFFSETS</a> -<li><a name="TOC13" href="#SEC13">NAMED SUBPATTERNS</a> -<li><a name="TOC14" href="#SEC14">OPTION NAMES</a> -<li><a name="TOC15" href="#SEC15">CHARACTER CODES</a> -<li><a name="TOC16" href="#SEC16">ERROR NAMES</a> -<li><a name="TOC17" href="#SEC17">ERROR TEXTS</a> -<li><a name="TOC18" href="#SEC18">CALLOUTS</a> -<li><a name="TOC19" href="#SEC19">TESTING</a> -<li><a name="TOC20" href="#SEC20">NOT SUPPORTED IN 16-BIT MODE</a> -<li><a name="TOC21" href="#SEC21">AUTHOR</a> -<li><a name="TOC22" href="#SEC22">REVISION</a> -</ul> -<P> -<b>#include <pcre.h></b> -</P> -<br><a name="SEC1" href="#TOC1">PCRE 16-BIT API BASIC FUNCTIONS</a><br> -<P> -<b>pcre16 *pcre16_compile(PCRE_SPTR16 <i>pattern</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -<br> -<br> -<b>pcre16 *pcre16_compile2(PCRE_SPTR16 <i>pattern</i>, int <i>options</i>,</b> -<b> int *<i>errorcodeptr</i>,</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -<br> -<br> -<b>pcre16_extra *pcre16_study(const pcre16 *<i>code</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>);</b> -<br> -<br> -<b>void pcre16_free_study(pcre16_extra *<i>extra</i>);</b> -<br> -<br> -<b>int pcre16_exec(const pcre16 *<i>code</i>, const pcre16_extra *<i>extra</i>,</b> -<b> PCRE_SPTR16 <i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>);</b> -<br> -<br> -<b>int pcre16_dfa_exec(const pcre16 *<i>code</i>, const pcre16_extra *<i>extra</i>,</b> -<b> PCRE_SPTR16 <i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> -<b> int *<i>workspace</i>, int <i>wscount</i>);</b> -</P> -<br><a name="SEC2" href="#TOC1">PCRE 16-BIT API STRING EXTRACTION FUNCTIONS</a><br> -<P> -<b>int pcre16_copy_named_substring(const pcre16 *<i>code</i>,</b> -<b> PCRE_SPTR16 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, PCRE_SPTR16 <i>stringname</i>,</b> -<b> PCRE_UCHAR16 *<i>buffer</i>, int <i>buffersize</i>);</b> -<br> -<br> -<b>int pcre16_copy_substring(PCRE_SPTR16 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>, PCRE_UCHAR16 *<i>buffer</i>,</b> -<b> int <i>buffersize</i>);</b> -<br> -<br> -<b>int pcre16_get_named_substring(const pcre16 *<i>code</i>,</b> -<b> PCRE_SPTR16 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, PCRE_SPTR16 <i>stringname</i>,</b> -<b> PCRE_SPTR16 *<i>stringptr</i>);</b> -<br> -<br> -<b>int pcre16_get_stringnumber(const pcre16 *<i>code</i>,</b> -<b>" PCRE_SPTR16 <i>name</i>);</b> -<br> -<br> -<b>int pcre16_get_stringtable_entries(const pcre16 *<i>code</i>,</b> -<b> PCRE_SPTR16 <i>name</i>, PCRE_UCHAR16 **<i>first</i>, PCRE_UCHAR16 **<i>last</i>);</b> -<br> -<br> -<b>int pcre16_get_substring(PCRE_SPTR16 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>,</b> -<b> PCRE_SPTR16 *<i>stringptr</i>);</b> -<br> -<br> -<b>int pcre16_get_substring_list(PCRE_SPTR16 <i>subject</i>,</b> -<b> int *<i>ovector</i>, int <i>stringcount</i>, PCRE_SPTR16 **<i>listptr</i>);</b> -<br> -<br> -<b>void pcre16_free_substring(PCRE_SPTR16 <i>stringptr</i>);</b> -<br> -<br> -<b>void pcre16_free_substring_list(PCRE_SPTR16 *<i>stringptr</i>);</b> -</P> -<br><a name="SEC3" href="#TOC1">PCRE 16-BIT API AUXILIARY FUNCTIONS</a><br> -<P> -<b>pcre16_jit_stack *pcre16_jit_stack_alloc(int <i>startsize</i>, int <i>maxsize</i>);</b> -<br> -<br> -<b>void pcre16_jit_stack_free(pcre16_jit_stack *<i>stack</i>);</b> -<br> -<br> -<b>void pcre16_assign_jit_stack(pcre16_extra *<i>extra</i>,</b> -<b> pcre16_jit_callback <i>callback</i>, void *<i>data</i>);</b> -<br> -<br> -<b>const unsigned char *pcre16_maketables(void);</b> -<br> -<br> -<b>int pcre16_fullinfo(const pcre16 *<i>code</i>, const pcre16_extra *<i>extra</i>,</b> -<b> int <i>what</i>, void *<i>where</i>);</b> -<br> -<br> -<b>int pcre16_refcount(pcre16 *<i>code</i>, int <i>adjust</i>);</b> -<br> -<br> -<b>int pcre16_config(int <i>what</i>, void *<i>where</i>);</b> -<br> -<br> -<b>const char *pcre16_version(void);</b> -<br> -<br> -<b>int pcre16_pattern_to_host_byte_order(pcre16 *<i>code</i>,</b> -<b> pcre16_extra *<i>extra</i>, const unsigned char *<i>tables</i>);</b> -</P> -<br><a name="SEC4" href="#TOC1">PCRE 16-BIT API INDIRECTED FUNCTIONS</a><br> -<P> -<b>void *(*pcre16_malloc)(size_t);</b> -<br> -<br> -<b>void (*pcre16_free)(void *);</b> -<br> -<br> -<b>void *(*pcre16_stack_malloc)(size_t);</b> -<br> -<br> -<b>void (*pcre16_stack_free)(void *);</b> -<br> -<br> -<b>int (*pcre16_callout)(pcre16_callout_block *);</b> -</P> -<br><a name="SEC5" href="#TOC1">PCRE 16-BIT API 16-BIT-ONLY FUNCTION</a><br> -<P> -<b>int pcre16_utf16_to_host_byte_order(PCRE_UCHAR16 *<i>output</i>,</b> -<b> PCRE_SPTR16 <i>input</i>, int <i>length</i>, int *<i>byte_order</i>,</b> -<b> int <i>keep_boms</i>);</b> -</P> -<br><a name="SEC6" href="#TOC1">THE PCRE 16-BIT LIBRARY</a><br> -<P> -Starting with release 8.30, it is possible to compile a PCRE library that -supports 16-bit character strings, including UTF-16 strings, as well as or -instead of the original 8-bit library. The majority of the work to make this -possible was done by Zoltan Herczeg. The two libraries contain identical sets -of functions, used in exactly the same way. Only the names of the functions and -the data types of their arguments and results are different. To avoid -over-complication and reduce the documentation maintenance load, most of the -PCRE documentation describes the 8-bit library, with only occasional references -to the 16-bit library. This page describes what is different when you use the -16-bit library. -</P> -<P> -WARNING: A single application can be linked with both libraries, but you must -take care when processing any particular pattern to use functions from just one -library. For example, if you want to study a pattern that was compiled with -<b>pcre16_compile()</b>, you must do so with <b>pcre16_study()</b>, not -<b>pcre_study()</b>, and you must free the study data with -<b>pcre16_free_study()</b>. -</P> -<br><a name="SEC7" href="#TOC1">THE HEADER FILE</a><br> -<P> -There is only one header file, <b>pcre.h</b>. It contains prototypes for all the -functions in all libraries, as well as definitions of flags, structures, error -codes, etc. -</P> -<br><a name="SEC8" href="#TOC1">THE LIBRARY NAME</a><br> -<P> -In Unix-like systems, the 16-bit library is called <b>libpcre16</b>, and can -normally be accesss by adding <b>-lpcre16</b> to the command for linking an -application that uses PCRE. -</P> -<br><a name="SEC9" href="#TOC1">STRING TYPES</a><br> -<P> -In the 8-bit library, strings are passed to PCRE library functions as vectors -of bytes with the C type "char *". In the 16-bit library, strings are passed as -vectors of unsigned 16-bit quantities. The macro PCRE_UCHAR16 specifies an -appropriate data type, and PCRE_SPTR16 is defined as "const PCRE_UCHAR16 *". In -very many environments, "short int" is a 16-bit data type. When PCRE is built, -it defines PCRE_UCHAR16 as "unsigned short int", but checks that it really is a -16-bit data type. If it is not, the build fails with an error message telling -the maintainer to modify the definition appropriately. -</P> -<br><a name="SEC10" href="#TOC1">STRUCTURE TYPES</a><br> -<P> -The types of the opaque structures that are used for compiled 16-bit patterns -and JIT stacks are <b>pcre16</b> and <b>pcre16_jit_stack</b> respectively. The -type of the user-accessible structure that is returned by <b>pcre16_study()</b> -is <b>pcre16_extra</b>, and the type of the structure that is used for passing -data to a callout function is <b>pcre16_callout_block</b>. These structures -contain the same fields, with the same names, as their 8-bit counterparts. The -only difference is that pointers to character strings are 16-bit instead of -8-bit types. -</P> -<br><a name="SEC11" href="#TOC1">16-BIT FUNCTIONS</a><br> -<P> -For every function in the 8-bit library there is a corresponding function in -the 16-bit library with a name that starts with <b>pcre16_</b> instead of -<b>pcre_</b>. The prototypes are listed above. In addition, there is one extra -function, <b>pcre16_utf16_to_host_byte_order()</b>. This is a utility function -that converts a UTF-16 character string to host byte order if necessary. The -other 16-bit functions expect the strings they are passed to be in host byte -order. -</P> -<P> -The <i>input</i> and <i>output</i> arguments of -<b>pcre16_utf16_to_host_byte_order()</b> may point to the same address, that is, -conversion in place is supported. The output buffer must be at least as long as -the input. -</P> -<P> -The <i>length</i> argument specifies the number of 16-bit data units in the -input string; a negative value specifies a zero-terminated string. -</P> -<P> -If <i>byte_order</i> is NULL, it is assumed that the string starts off in host -byte order. This may be changed by byte-order marks (BOMs) anywhere in the -string (commonly as the first character). -</P> -<P> -If <i>byte_order</i> is not NULL, a non-zero value of the integer to which it -points means that the input starts off in host byte order, otherwise the -opposite order is assumed. Again, BOMs in the string can change this. The final -byte order is passed back at the end of processing. -</P> -<P> -If <i>keep_boms</i> is not zero, byte-order mark characters (0xfeff) are copied -into the output string. Otherwise they are discarded. -</P> -<P> -The result of the function is the number of 16-bit units placed into the output -buffer, including the zero terminator if the string was zero-terminated. -</P> -<br><a name="SEC12" href="#TOC1">SUBJECT STRING OFFSETS</a><br> -<P> -The lengths and starting offsets of subject strings must be specified in 16-bit -data units, and the offsets within subject strings that are returned by the -matching functions are in also 16-bit units rather than bytes. -</P> -<br><a name="SEC13" href="#TOC1">NAMED SUBPATTERNS</a><br> -<P> -The name-to-number translation table that is maintained for named subpatterns -uses 16-bit characters. The <b>pcre16_get_stringtable_entries()</b> function -returns the length of each entry in the table as the number of 16-bit data -units. -</P> -<br><a name="SEC14" href="#TOC1">OPTION NAMES</a><br> -<P> -There are two new general option names, PCRE_UTF16 and PCRE_NO_UTF16_CHECK, -which correspond to PCRE_UTF8 and PCRE_NO_UTF8_CHECK in the 8-bit library. In -fact, these new options define the same bits in the options word. There is a -discussion about the -<a href="pcreunicode.html#utf16strings">validity of UTF-16 strings</a> -in the -<a href="pcreunicode.html"><b>pcreunicode</b></a> -page. -</P> -<P> -For the <b>pcre16_config()</b> function there is an option PCRE_CONFIG_UTF16 -that returns 1 if UTF-16 support is configured, otherwise 0. If this option is -given to <b>pcre_config()</b> or <b>pcre32_config()</b>, or if the -PCRE_CONFIG_UTF8 or PCRE_CONFIG_UTF32 option is given to <b>pcre16_config()</b>, -the result is the PCRE_ERROR_BADOPTION error. -</P> -<br><a name="SEC15" href="#TOC1">CHARACTER CODES</a><br> -<P> -In 16-bit mode, when PCRE_UTF16 is not set, character values are treated in the -same way as in 8-bit, non UTF-8 mode, except, of course, that they can range -from 0 to 0xffff instead of 0 to 0xff. Character types for characters less than -0xff can therefore be influenced by the locale in the same way as before. -Characters greater than 0xff have only one case, and no "type" (such as letter -or digit). -</P> -<P> -In UTF-16 mode, the character code is Unicode, in the range 0 to 0x10ffff, with -the exception of values in the range 0xd800 to 0xdfff because those are -"surrogate" values that are used in pairs to encode values greater than 0xffff. -</P> -<P> -A UTF-16 string can indicate its endianness by special code knows as a -byte-order mark (BOM). The PCRE functions do not handle this, expecting strings -to be in host byte order. A utility function called -<b>pcre16_utf16_to_host_byte_order()</b> is provided to help with this (see -above). -</P> -<br><a name="SEC16" href="#TOC1">ERROR NAMES</a><br> -<P> -The errors PCRE_ERROR_BADUTF16_OFFSET and PCRE_ERROR_SHORTUTF16 correspond to -their 8-bit counterparts. The error PCRE_ERROR_BADMODE is given when a compiled -pattern is passed to a function that processes patterns in the other -mode, for example, if a pattern compiled with <b>pcre_compile()</b> is passed to -<b>pcre16_exec()</b>. -</P> -<P> -There are new error codes whose names begin with PCRE_UTF16_ERR for invalid -UTF-16 strings, corresponding to the PCRE_UTF8_ERR codes for UTF-8 strings that -are described in the section entitled -<a href="pcreapi.html#badutf8reasons">"Reason codes for invalid UTF-8 strings"</a> -in the main -<a href="pcreapi.html"><b>pcreapi</b></a> -page. The UTF-16 errors are: -<pre> - PCRE_UTF16_ERR1 Missing low surrogate at end of string - PCRE_UTF16_ERR2 Invalid low surrogate follows high surrogate - PCRE_UTF16_ERR3 Isolated low surrogate - PCRE_UTF16_ERR4 Non-character -</PRE> -</P> -<br><a name="SEC17" href="#TOC1">ERROR TEXTS</a><br> -<P> -If there is an error while compiling a pattern, the error text that is passed -back by <b>pcre16_compile()</b> or <b>pcre16_compile2()</b> is still an 8-bit -character string, zero-terminated. -</P> -<br><a name="SEC18" href="#TOC1">CALLOUTS</a><br> -<P> -The <i>subject</i> and <i>mark</i> fields in the callout block that is passed to -a callout function point to 16-bit vectors. -</P> -<br><a name="SEC19" href="#TOC1">TESTING</a><br> -<P> -The <b>pcretest</b> program continues to operate with 8-bit input and output -files, but it can be used for testing the 16-bit library. If it is run with the -command line option <b>-16</b>, patterns and subject strings are converted from -8-bit to 16-bit before being passed to PCRE, and the 16-bit library functions -are used instead of the 8-bit ones. Returned 16-bit strings are converted to -8-bit for output. If both the 8-bit and the 32-bit libraries were not compiled, -<b>pcretest</b> defaults to 16-bit and the <b>-16</b> option is ignored. -</P> -<P> -When PCRE is being built, the <b>RunTest</b> script that is called by "make -check" uses the <b>pcretest</b> <b>-C</b> option to discover which of the 8-bit, -16-bit and 32-bit libraries has been built, and runs the tests appropriately. -</P> -<br><a name="SEC20" href="#TOC1">NOT SUPPORTED IN 16-BIT MODE</a><br> -<P> -Not all the features of the 8-bit library are available with the 16-bit -library. The C++ and POSIX wrapper functions support only the 8-bit library, -and the <b>pcregrep</b> program is at present 8-bit only. -</P> -<br><a name="SEC21" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC22" href="#TOC1">REVISION</a><br> -<P> -Last updated: 12 May 2013 -<br> -Copyright © 1997-2013 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre32.html b/plugins/Pcre16/docs/doc/html/pcre32.html deleted file mode 100644 index f96876e750..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre32.html +++ /dev/null @@ -1,382 +0,0 @@ -<html> -<head> -<title>pcre32 specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre32 man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">PCRE 32-BIT API BASIC FUNCTIONS</a> -<li><a name="TOC2" href="#SEC2">PCRE 32-BIT API STRING EXTRACTION FUNCTIONS</a> -<li><a name="TOC3" href="#SEC3">PCRE 32-BIT API AUXILIARY FUNCTIONS</a> -<li><a name="TOC4" href="#SEC4">PCRE 32-BIT API INDIRECTED FUNCTIONS</a> -<li><a name="TOC5" href="#SEC5">PCRE 32-BIT API 32-BIT-ONLY FUNCTION</a> -<li><a name="TOC6" href="#SEC6">THE PCRE 32-BIT LIBRARY</a> -<li><a name="TOC7" href="#SEC7">THE HEADER FILE</a> -<li><a name="TOC8" href="#SEC8">THE LIBRARY NAME</a> -<li><a name="TOC9" href="#SEC9">STRING TYPES</a> -<li><a name="TOC10" href="#SEC10">STRUCTURE TYPES</a> -<li><a name="TOC11" href="#SEC11">32-BIT FUNCTIONS</a> -<li><a name="TOC12" href="#SEC12">SUBJECT STRING OFFSETS</a> -<li><a name="TOC13" href="#SEC13">NAMED SUBPATTERNS</a> -<li><a name="TOC14" href="#SEC14">OPTION NAMES</a> -<li><a name="TOC15" href="#SEC15">CHARACTER CODES</a> -<li><a name="TOC16" href="#SEC16">ERROR NAMES</a> -<li><a name="TOC17" href="#SEC17">ERROR TEXTS</a> -<li><a name="TOC18" href="#SEC18">CALLOUTS</a> -<li><a name="TOC19" href="#SEC19">TESTING</a> -<li><a name="TOC20" href="#SEC20">NOT SUPPORTED IN 32-BIT MODE</a> -<li><a name="TOC21" href="#SEC21">AUTHOR</a> -<li><a name="TOC22" href="#SEC22">REVISION</a> -</ul> -<P> -<b>#include <pcre.h></b> -</P> -<br><a name="SEC1" href="#TOC1">PCRE 32-BIT API BASIC FUNCTIONS</a><br> -<P> -<b>pcre32 *pcre32_compile(PCRE_SPTR32 <i>pattern</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -<br> -<br> -<b>pcre32 *pcre32_compile2(PCRE_SPTR32 <i>pattern</i>, int <i>options</i>,</b> -<b> int *<i>errorcodeptr</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -<br> -<br> -<b>pcre32_extra *pcre32_study(const pcre32 *<i>code</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>);</b> -<br> -<br> -<b>void pcre32_free_study(pcre32_extra *<i>extra</i>);</b> -<br> -<br> -<b>int pcre32_exec(const pcre32 *<i>code</i>, const pcre32_extra *<i>extra</i>,</b> -<b> PCRE_SPTR32 <i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>);</b> -<br> -<br> -<b>int pcre32_dfa_exec(const pcre32 *<i>code</i>, const pcre32_extra *<i>extra</i>,</b> -<b> PCRE_SPTR32 <i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> -<b> int *<i>workspace</i>, int <i>wscount</i>);</b> -</P> -<br><a name="SEC2" href="#TOC1">PCRE 32-BIT API STRING EXTRACTION FUNCTIONS</a><br> -<P> -<b>int pcre32_copy_named_substring(const pcre32 *<i>code</i>,</b> -<b> PCRE_SPTR32 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, PCRE_SPTR32 <i>stringname</i>,</b> -<b> PCRE_UCHAR32 *<i>buffer</i>, int <i>buffersize</i>);</b> -<br> -<br> -<b>int pcre32_copy_substring(PCRE_SPTR32 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>, PCRE_UCHAR32 *<i>buffer</i>,</b> -<b> int <i>buffersize</i>);</b> -<br> -<br> -<b>int pcre32_get_named_substring(const pcre32 *<i>code</i>,</b> -<b> PCRE_SPTR32 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, PCRE_SPTR32 <i>stringname</i>,</b> -<b> PCRE_SPTR32 *<i>stringptr</i>);</b> -<br> -<br> -<b>int pcre32_get_stringnumber(const pcre32 *<i>code</i>,</b> -<b> PCRE_SPTR32 <i>name</i>);</b> -<br> -<br> -<b>int pcre32_get_stringtable_entries(const pcre32 *<i>code</i>,</b> -<b> PCRE_SPTR32 <i>name</i>, PCRE_UCHAR32 **<i>first</i>, PCRE_UCHAR32 **<i>last</i>);</b> -<br> -<br> -<b>int pcre32_get_substring(PCRE_SPTR32 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>,</b> -<b> PCRE_SPTR32 *<i>stringptr</i>);</b> -<br> -<br> -<b>int pcre32_get_substring_list(PCRE_SPTR32 <i>subject</i>,</b> -<b> int *<i>ovector</i>, int <i>stringcount</i>, PCRE_SPTR32 **<i>listptr</i>);</b> -<br> -<br> -<b>void pcre32_free_substring(PCRE_SPTR32 <i>stringptr</i>);</b> -<br> -<br> -<b>void pcre32_free_substring_list(PCRE_SPTR32 *<i>stringptr</i>);</b> -</P> -<br><a name="SEC3" href="#TOC1">PCRE 32-BIT API AUXILIARY FUNCTIONS</a><br> -<P> -<b>pcre32_jit_stack *pcre32_jit_stack_alloc(int <i>startsize</i>, int <i>maxsize</i>);</b> -<br> -<br> -<b>void pcre32_jit_stack_free(pcre32_jit_stack *<i>stack</i>);</b> -<br> -<br> -<b>void pcre32_assign_jit_stack(pcre32_extra *<i>extra</i>,</b> -<b> pcre32_jit_callback <i>callback</i>, void *<i>data</i>);</b> -<br> -<br> -<b>const unsigned char *pcre32_maketables(void);</b> -<br> -<br> -<b>int pcre32_fullinfo(const pcre32 *<i>code</i>, const pcre32_extra *<i>extra</i>,</b> -<b> int <i>what</i>, void *<i>where</i>);</b> -<br> -<br> -<b>int pcre32_refcount(pcre32 *<i>code</i>, int <i>adjust</i>);</b> -<br> -<br> -<b>int pcre32_config(int <i>what</i>, void *<i>where</i>);</b> -<br> -<br> -<b>const char *pcre32_version(void);</b> -<br> -<br> -<b>int pcre32_pattern_to_host_byte_order(pcre32 *<i>code</i>,</b> -<b> pcre32_extra *<i>extra</i>, const unsigned char *<i>tables</i>);</b> -</P> -<br><a name="SEC4" href="#TOC1">PCRE 32-BIT API INDIRECTED FUNCTIONS</a><br> -<P> -<b>void *(*pcre32_malloc)(size_t);</b> -<br> -<br> -<b>void (*pcre32_free)(void *);</b> -<br> -<br> -<b>void *(*pcre32_stack_malloc)(size_t);</b> -<br> -<br> -<b>void (*pcre32_stack_free)(void *);</b> -<br> -<br> -<b>int (*pcre32_callout)(pcre32_callout_block *);</b> -</P> -<br><a name="SEC5" href="#TOC1">PCRE 32-BIT API 32-BIT-ONLY FUNCTION</a><br> -<P> -<b>int pcre32_utf32_to_host_byte_order(PCRE_UCHAR32 *<i>output</i>,</b> -<b> PCRE_SPTR32 <i>input</i>, int <i>length</i>, int *<i>byte_order</i>,</b> -<b> int <i>keep_boms</i>);</b> -</P> -<br><a name="SEC6" href="#TOC1">THE PCRE 32-BIT LIBRARY</a><br> -<P> -Starting with release 8.32, it is possible to compile a PCRE library that -supports 32-bit character strings, including UTF-32 strings, as well as or -instead of the original 8-bit library. This work was done by Christian Persch, -based on the work done by Zoltan Herczeg for the 16-bit library. All three -libraries contain identical sets of functions, used in exactly the same way. -Only the names of the functions and the data types of their arguments and -results are different. To avoid over-complication and reduce the documentation -maintenance load, most of the PCRE documentation describes the 8-bit library, -with only occasional references to the 16-bit and 32-bit libraries. This page -describes what is different when you use the 32-bit library. -</P> -<P> -WARNING: A single application can be linked with all or any of the three -libraries, but you must take care when processing any particular pattern -to use functions from just one library. For example, if you want to study -a pattern that was compiled with <b>pcre32_compile()</b>, you must do so -with <b>pcre32_study()</b>, not <b>pcre_study()</b>, and you must free the -study data with <b>pcre32_free_study()</b>. -</P> -<br><a name="SEC7" href="#TOC1">THE HEADER FILE</a><br> -<P> -There is only one header file, <b>pcre.h</b>. It contains prototypes for all the -functions in all libraries, as well as definitions of flags, structures, error -codes, etc. -</P> -<br><a name="SEC8" href="#TOC1">THE LIBRARY NAME</a><br> -<P> -In Unix-like systems, the 32-bit library is called <b>libpcre32</b>, and can -normally be accesss by adding <b>-lpcre32</b> to the command for linking an -application that uses PCRE. -</P> -<br><a name="SEC9" href="#TOC1">STRING TYPES</a><br> -<P> -In the 8-bit library, strings are passed to PCRE library functions as vectors -of bytes with the C type "char *". In the 32-bit library, strings are passed as -vectors of unsigned 32-bit quantities. The macro PCRE_UCHAR32 specifies an -appropriate data type, and PCRE_SPTR32 is defined as "const PCRE_UCHAR32 *". In -very many environments, "unsigned int" is a 32-bit data type. When PCRE is -built, it defines PCRE_UCHAR32 as "unsigned int", but checks that it really is -a 32-bit data type. If it is not, the build fails with an error message telling -the maintainer to modify the definition appropriately. -</P> -<br><a name="SEC10" href="#TOC1">STRUCTURE TYPES</a><br> -<P> -The types of the opaque structures that are used for compiled 32-bit patterns -and JIT stacks are <b>pcre32</b> and <b>pcre32_jit_stack</b> respectively. The -type of the user-accessible structure that is returned by <b>pcre32_study()</b> -is <b>pcre32_extra</b>, and the type of the structure that is used for passing -data to a callout function is <b>pcre32_callout_block</b>. These structures -contain the same fields, with the same names, as their 8-bit counterparts. The -only difference is that pointers to character strings are 32-bit instead of -8-bit types. -</P> -<br><a name="SEC11" href="#TOC1">32-BIT FUNCTIONS</a><br> -<P> -For every function in the 8-bit library there is a corresponding function in -the 32-bit library with a name that starts with <b>pcre32_</b> instead of -<b>pcre_</b>. The prototypes are listed above. In addition, there is one extra -function, <b>pcre32_utf32_to_host_byte_order()</b>. This is a utility function -that converts a UTF-32 character string to host byte order if necessary. The -other 32-bit functions expect the strings they are passed to be in host byte -order. -</P> -<P> -The <i>input</i> and <i>output</i> arguments of -<b>pcre32_utf32_to_host_byte_order()</b> may point to the same address, that is, -conversion in place is supported. The output buffer must be at least as long as -the input. -</P> -<P> -The <i>length</i> argument specifies the number of 32-bit data units in the -input string; a negative value specifies a zero-terminated string. -</P> -<P> -If <i>byte_order</i> is NULL, it is assumed that the string starts off in host -byte order. This may be changed by byte-order marks (BOMs) anywhere in the -string (commonly as the first character). -</P> -<P> -If <i>byte_order</i> is not NULL, a non-zero value of the integer to which it -points means that the input starts off in host byte order, otherwise the -opposite order is assumed. Again, BOMs in the string can change this. The final -byte order is passed back at the end of processing. -</P> -<P> -If <i>keep_boms</i> is not zero, byte-order mark characters (0xfeff) are copied -into the output string. Otherwise they are discarded. -</P> -<P> -The result of the function is the number of 32-bit units placed into the output -buffer, including the zero terminator if the string was zero-terminated. -</P> -<br><a name="SEC12" href="#TOC1">SUBJECT STRING OFFSETS</a><br> -<P> -The lengths and starting offsets of subject strings must be specified in 32-bit -data units, and the offsets within subject strings that are returned by the -matching functions are in also 32-bit units rather than bytes. -</P> -<br><a name="SEC13" href="#TOC1">NAMED SUBPATTERNS</a><br> -<P> -The name-to-number translation table that is maintained for named subpatterns -uses 32-bit characters. The <b>pcre32_get_stringtable_entries()</b> function -returns the length of each entry in the table as the number of 32-bit data -units. -</P> -<br><a name="SEC14" href="#TOC1">OPTION NAMES</a><br> -<P> -There are two new general option names, PCRE_UTF32 and PCRE_NO_UTF32_CHECK, -which correspond to PCRE_UTF8 and PCRE_NO_UTF8_CHECK in the 8-bit library. In -fact, these new options define the same bits in the options word. There is a -discussion about the -<a href="pcreunicode.html#utf32strings">validity of UTF-32 strings</a> -in the -<a href="pcreunicode.html"><b>pcreunicode</b></a> -page. -</P> -<P> -For the <b>pcre32_config()</b> function there is an option PCRE_CONFIG_UTF32 -that returns 1 if UTF-32 support is configured, otherwise 0. If this option is -given to <b>pcre_config()</b> or <b>pcre16_config()</b>, or if the -PCRE_CONFIG_UTF8 or PCRE_CONFIG_UTF16 option is given to <b>pcre32_config()</b>, -the result is the PCRE_ERROR_BADOPTION error. -</P> -<br><a name="SEC15" href="#TOC1">CHARACTER CODES</a><br> -<P> -In 32-bit mode, when PCRE_UTF32 is not set, character values are treated in the -same way as in 8-bit, non UTF-8 mode, except, of course, that they can range -from 0 to 0x7fffffff instead of 0 to 0xff. Character types for characters less -than 0xff can therefore be influenced by the locale in the same way as before. -Characters greater than 0xff have only one case, and no "type" (such as letter -or digit). -</P> -<P> -In UTF-32 mode, the character code is Unicode, in the range 0 to 0x10ffff, with -the exception of values in the range 0xd800 to 0xdfff because those are -"surrogate" values that are ill-formed in UTF-32. -</P> -<P> -A UTF-32 string can indicate its endianness by special code knows as a -byte-order mark (BOM). The PCRE functions do not handle this, expecting strings -to be in host byte order. A utility function called -<b>pcre32_utf32_to_host_byte_order()</b> is provided to help with this (see -above). -</P> -<br><a name="SEC16" href="#TOC1">ERROR NAMES</a><br> -<P> -The error PCRE_ERROR_BADUTF32 corresponds to its 8-bit counterpart. -The error PCRE_ERROR_BADMODE is given when a compiled -pattern is passed to a function that processes patterns in the other -mode, for example, if a pattern compiled with <b>pcre_compile()</b> is passed to -<b>pcre32_exec()</b>. -</P> -<P> -There are new error codes whose names begin with PCRE_UTF32_ERR for invalid -UTF-32 strings, corresponding to the PCRE_UTF8_ERR codes for UTF-8 strings that -are described in the section entitled -<a href="pcreapi.html#badutf8reasons">"Reason codes for invalid UTF-8 strings"</a> -in the main -<a href="pcreapi.html"><b>pcreapi</b></a> -page. The UTF-32 errors are: -<pre> - PCRE_UTF32_ERR1 Surrogate character (range from 0xd800 to 0xdfff) - PCRE_UTF32_ERR2 Non-character - PCRE_UTF32_ERR3 Character > 0x10ffff -</PRE> -</P> -<br><a name="SEC17" href="#TOC1">ERROR TEXTS</a><br> -<P> -If there is an error while compiling a pattern, the error text that is passed -back by <b>pcre32_compile()</b> or <b>pcre32_compile2()</b> is still an 8-bit -character string, zero-terminated. -</P> -<br><a name="SEC18" href="#TOC1">CALLOUTS</a><br> -<P> -The <i>subject</i> and <i>mark</i> fields in the callout block that is passed to -a callout function point to 32-bit vectors. -</P> -<br><a name="SEC19" href="#TOC1">TESTING</a><br> -<P> -The <b>pcretest</b> program continues to operate with 8-bit input and output -files, but it can be used for testing the 32-bit library. If it is run with the -command line option <b>-32</b>, patterns and subject strings are converted from -8-bit to 32-bit before being passed to PCRE, and the 32-bit library functions -are used instead of the 8-bit ones. Returned 32-bit strings are converted to -8-bit for output. If both the 8-bit and the 16-bit libraries were not compiled, -<b>pcretest</b> defaults to 32-bit and the <b>-32</b> option is ignored. -</P> -<P> -When PCRE is being built, the <b>RunTest</b> script that is called by "make -check" uses the <b>pcretest</b> <b>-C</b> option to discover which of the 8-bit, -16-bit and 32-bit libraries has been built, and runs the tests appropriately. -</P> -<br><a name="SEC20" href="#TOC1">NOT SUPPORTED IN 32-BIT MODE</a><br> -<P> -Not all the features of the 8-bit library are available with the 32-bit -library. The C++ and POSIX wrapper functions support only the 8-bit library, -and the <b>pcregrep</b> program is at present 8-bit only. -</P> -<br><a name="SEC21" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC22" href="#TOC1">REVISION</a><br> -<P> -Last updated: 12 May 2013 -<br> -Copyright © 1997-2013 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_assign_jit_stack.html b/plugins/Pcre16/docs/doc/html/pcre_assign_jit_stack.html deleted file mode 100644 index b2eef704db..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_assign_jit_stack.html +++ /dev/null @@ -1,76 +0,0 @@ -<html> -<head> -<title>pcre_assign_jit_stack specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_assign_jit_stack man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>void pcre_assign_jit_stack(pcre_extra *<i>extra</i>,</b> -<b> pcre_jit_callback <i>callback</i>, void *<i>data</i>);</b> -<br> -<br> -<b>void pcre16_assign_jit_stack(pcre16_extra *<i>extra</i>,</b> -<b> pcre16_jit_callback <i>callback</i>, void *<i>data</i>);</b> -<br> -<br> -<b>void pcre32_assign_jit_stack(pcre32_extra *<i>extra</i>,</b> -<b> pcre32_jit_callback <i>callback</i>, void *<i>data</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function provides control over the memory used as a stack at run-time by a -call to <b>pcre[16|32]_exec()</b> with a pattern that has been successfully -compiled with JIT optimization. The arguments are: -<pre> - extra the data pointer returned by <b>pcre[16|32]_study()</b> - callback a callback function - data a JIT stack or a value to be passed to the callback - function -</PRE> -</P> -<P> -If <i>callback</i> is NULL and <i>data</i> is NULL, an internal 32K block on -the machine stack is used. -</P> -<P> -If <i>callback</i> is NULL and <i>data</i> is not NULL, <i>data</i> must -be a valid JIT stack, the result of calling <b>pcre[16|32]_jit_stack_alloc()</b>. -</P> -<P> -If <i>callback</i> not NULL, it is called with <i>data</i> as an argument at -the start of matching, in order to set up a JIT stack. If the result is NULL, -the internal 32K stack is used; otherwise the return value must be a valid JIT -stack, the result of calling <b>pcre[16|32]_jit_stack_alloc()</b>. -</P> -<P> -You may safely assign the same JIT stack to multiple patterns, as long as they -are all matched in the same thread. In a multithread application, each thread -must use its own JIT stack. For more details, see the -<a href="pcrejit.html"><b>pcrejit</b></a> -page. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_compile.html b/plugins/Pcre16/docs/doc/html/pcre_compile.html deleted file mode 100644 index 95b4bec63c..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_compile.html +++ /dev/null @@ -1,111 +0,0 @@ -<html> -<head> -<title>pcre_compile specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_compile man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>pcre *pcre_compile(const char *<i>pattern</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -<br> -<br> -<b>pcre16 *pcre16_compile(PCRE_SPTR16 <i>pattern</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -<br> -<br> -<b>pcre32 *pcre32_compile(PCRE_SPTR32 <i>pattern</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function compiles a regular expression into an internal form. It is the -same as <b>pcre[16|32]_compile2()</b>, except for the absence of the -<i>errorcodeptr</i> argument. Its arguments are: -<pre> - <i>pattern</i> A zero-terminated string containing the - regular expression to be compiled - <i>options</i> Zero or more option bits - <i>errptr</i> Where to put an error message - <i>erroffset</i> Offset in pattern where error was found - <i>tableptr</i> Pointer to character tables, or NULL to - use the built-in default -</pre> -The option bits are: -<pre> - PCRE_ANCHORED Force pattern anchoring - PCRE_AUTO_CALLOUT Compile automatic callouts - PCRE_BSR_ANYCRLF \R matches only CR, LF, or CRLF - PCRE_BSR_UNICODE \R matches all Unicode line endings - PCRE_CASELESS Do caseless matching - PCRE_DOLLAR_ENDONLY $ not to match newline at end - PCRE_DOTALL . matches anything including NL - PCRE_DUPNAMES Allow duplicate names for subpatterns - PCRE_EXTENDED Ignore white space and # comments - PCRE_EXTRA PCRE extra features - (not much use currently) - PCRE_FIRSTLINE Force matching to be before newline - PCRE_JAVASCRIPT_COMPAT JavaScript compatibility - PCRE_MULTILINE ^ and $ match newlines within data - PCRE_NEVER_UTF Lock out UTF, e.g. via (*UTF) - PCRE_NEWLINE_ANY Recognize any Unicode newline sequence - PCRE_NEWLINE_ANYCRLF Recognize CR, LF, and CRLF as newline - sequences - PCRE_NEWLINE_CR Set CR as the newline sequence - PCRE_NEWLINE_CRLF Set CRLF as the newline sequence - PCRE_NEWLINE_LF Set LF as the newline sequence - PCRE_NO_AUTO_CAPTURE Disable numbered capturing paren- - theses (named ones available) - PCRE_NO_AUTO_POSSESS Disable auto-possessification - PCRE_NO_START_OPTIMIZE Disable match-time start optimizations - PCRE_NO_UTF16_CHECK Do not check the pattern for UTF-16 - validity (only relevant if - PCRE_UTF16 is set) - PCRE_NO_UTF32_CHECK Do not check the pattern for UTF-32 - validity (only relevant if - PCRE_UTF32 is set) - PCRE_NO_UTF8_CHECK Do not check the pattern for UTF-8 - validity (only relevant if - PCRE_UTF8 is set) - PCRE_UCP Use Unicode properties for \d, \w, etc. - PCRE_UNGREEDY Invert greediness of quantifiers - PCRE_UTF16 Run in <b>pcre16_compile()</b> UTF-16 mode - PCRE_UTF32 Run in <b>pcre32_compile()</b> UTF-32 mode - PCRE_UTF8 Run in <b>pcre_compile()</b> UTF-8 mode -</pre> -PCRE must be built with UTF support in order to use PCRE_UTF8/16/32 and -PCRE_NO_UTF8/16/32_CHECK, and with UCP support if PCRE_UCP is used. -</P> -<P> -The yield of the function is a pointer to a private data structure that -contains the compiled pattern, or NULL if an error was detected. Note that -compiling regular expressions with one version of PCRE for use with a different -version is not guaranteed to work and may cause crashes. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_compile2.html b/plugins/Pcre16/docs/doc/html/pcre_compile2.html deleted file mode 100644 index 9cd56a237b..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_compile2.html +++ /dev/null @@ -1,115 +0,0 @@ -<html> -<head> -<title>pcre_compile2 specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_compile2 man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>pcre *pcre_compile2(const char *<i>pattern</i>, int <i>options</i>,</b> -<b> int *<i>errorcodeptr</i>,</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -<br> -<br> -<b>pcre16 *pcre16_compile2(PCRE_SPTR16 <i>pattern</i>, int <i>options</i>,</b> -<b> int *<i>errorcodeptr</i>,</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -<br> -<br> -<b>pcre32 *pcre32_compile2(PCRE_SPTR32 <i>pattern</i>, int <i>options</i>,</b> -<b>" int *<i>errorcodeptr</i>,£</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function compiles a regular expression into an internal form. It is the -same as <b>pcre[16|32]_compile()</b>, except for the addition of the -<i>errorcodeptr</i> argument. The arguments are: -<pre> - <i>pattern</i> A zero-terminated string containing the - regular expression to be compiled - <i>options</i> Zero or more option bits - <i>errorcodeptr</i> Where to put an error code - <i>errptr</i> Where to put an error message - <i>erroffset</i> Offset in pattern where error was found - <i>tableptr</i> Pointer to character tables, or NULL to - use the built-in default -</pre> -The option bits are: -<pre> - PCRE_ANCHORED Force pattern anchoring - PCRE_AUTO_CALLOUT Compile automatic callouts - PCRE_BSR_ANYCRLF \R matches only CR, LF, or CRLF - PCRE_BSR_UNICODE \R matches all Unicode line endings - PCRE_CASELESS Do caseless matching - PCRE_DOLLAR_ENDONLY $ not to match newline at end - PCRE_DOTALL . matches anything including NL - PCRE_DUPNAMES Allow duplicate names for subpatterns - PCRE_EXTENDED Ignore white space and # comments - PCRE_EXTRA PCRE extra features - (not much use currently) - PCRE_FIRSTLINE Force matching to be before newline - PCRE_JAVASCRIPT_COMPAT JavaScript compatibility - PCRE_MULTILINE ^ and $ match newlines within data - PCRE_NEVER_UTF Lock out UTF, e.g. via (*UTF) - PCRE_NEWLINE_ANY Recognize any Unicode newline sequence - PCRE_NEWLINE_ANYCRLF Recognize CR, LF, and CRLF as newline - sequences - PCRE_NEWLINE_CR Set CR as the newline sequence - PCRE_NEWLINE_CRLF Set CRLF as the newline sequence - PCRE_NEWLINE_LF Set LF as the newline sequence - PCRE_NO_AUTO_CAPTURE Disable numbered capturing paren- - theses (named ones available) - PCRE_NO_AUTO_POSSESS Disable auto-possessification - PCRE_NO_START_OPTIMIZE Disable match-time start optimizations - PCRE_NO_UTF16_CHECK Do not check the pattern for UTF-16 - validity (only relevant if - PCRE_UTF16 is set) - PCRE_NO_UTF32_CHECK Do not check the pattern for UTF-32 - validity (only relevant if - PCRE_UTF32 is set) - PCRE_NO_UTF8_CHECK Do not check the pattern for UTF-8 - validity (only relevant if - PCRE_UTF8 is set) - PCRE_UCP Use Unicode properties for \d, \w, etc. - PCRE_UNGREEDY Invert greediness of quantifiers - PCRE_UTF16 Run <b>pcre16_compile()</b> in UTF-16 mode - PCRE_UTF32 Run <b>pcre32_compile()</b> in UTF-32 mode - PCRE_UTF8 Run <b>pcre_compile()</b> in UTF-8 mode -</pre> -PCRE must be built with UTF support in order to use PCRE_UTF8/16/32 and -PCRE_NO_UTF8/16/32_CHECK, and with UCP support if PCRE_UCP is used. -</P> -<P> -The yield of the function is a pointer to a private data structure that -contains the compiled pattern, or NULL if an error was detected. Note that -compiling regular expressions with one version of PCRE for use with a different -version is not guaranteed to work and may cause crashes. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_config.html b/plugins/Pcre16/docs/doc/html/pcre_config.html deleted file mode 100644 index bcdcdded70..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_config.html +++ /dev/null @@ -1,92 +0,0 @@ -<html> -<head> -<title>pcre_config specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_config man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_config(int <i>what</i>, void *<i>where</i>);</b> -</P> -<P> -<b>int pcre16_config(int <i>what</i>, void *<i>where</i>);</b> -</P> -<P> -<b>int pcre32_config(int <i>what</i>, void *<i>where</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function makes it possible for a client program to find out which optional -features are available in the version of the PCRE library it is using. The -arguments are as follows: -<pre> - <i>what</i> A code specifying what information is required - <i>where</i> Points to where to put the data -</pre> -The <i>where</i> argument must point to an integer variable, except for -PCRE_CONFIG_MATCH_LIMIT and PCRE_CONFIG_MATCH_LIMIT_RECURSION, when it must -point to an unsigned long integer. The available codes are: -<pre> - PCRE_CONFIG_JIT Availability of just-in-time compiler - support (1=yes 0=no) - PCRE_CONFIG_JITTARGET String containing information about the - target architecture for the JIT compiler, - or NULL if there is no JIT support - PCRE_CONFIG_LINK_SIZE Internal link size: 2, 3, or 4 - PCRE_CONFIG_PARENS_LIMIT Parentheses nesting limit - PCRE_CONFIG_MATCH_LIMIT Internal resource limit - PCRE_CONFIG_MATCH_LIMIT_RECURSION - Internal recursion depth limit - PCRE_CONFIG_NEWLINE Value of the default newline sequence: - 13 (0x000d) for CR - 10 (0x000a) for LF - 3338 (0x0d0a) for CRLF - -2 for ANYCRLF - -1 for ANY - PCRE_CONFIG_BSR Indicates what \R matches by default: - 0 all Unicode line endings - 1 CR, LF, or CRLF only - PCRE_CONFIG_POSIX_MALLOC_THRESHOLD - Threshold of return slots, above which - <b>malloc()</b> is used by the POSIX API - PCRE_CONFIG_STACKRECURSE Recursion implementation (1=stack 0=heap) - PCRE_CONFIG_UTF16 Availability of UTF-16 support (1=yes - 0=no); option for <b>pcre16_config()</b> - PCRE_CONFIG_UTF32 Availability of UTF-32 support (1=yes - 0=no); option for <b>pcre32_config()</b> - PCRE_CONFIG_UTF8 Availability of UTF-8 support (1=yes 0=no); - option for <b>pcre_config()</b> - PCRE_CONFIG_UNICODE_PROPERTIES - Availability of Unicode property support - (1=yes 0=no) -</pre> -The function yields 0 on success or PCRE_ERROR_BADOPTION otherwise. That error -is also given if PCRE_CONFIG_UTF16 or PCRE_CONFIG_UTF32 is passed to -<b>pcre_config()</b>, if PCRE_CONFIG_UTF8 or PCRE_CONFIG_UTF32 is passed to -<b>pcre16_config()</b>, or if PCRE_CONFIG_UTF8 or PCRE_CONFIG_UTF16 is passed to -<b>pcre32_config()</b>. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_copy_named_substring.html b/plugins/Pcre16/docs/doc/html/pcre_copy_named_substring.html deleted file mode 100644 index 77b48043cd..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_copy_named_substring.html +++ /dev/null @@ -1,65 +0,0 @@ -<html> -<head> -<title>pcre_copy_named_substring specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_copy_named_substring man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_copy_named_substring(const pcre *<i>code</i>,</b> -<b> const char *<i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, const char *<i>stringname</i>,</b> -<b> char *<i>buffer</i>, int <i>buffersize</i>);</b> -<br> -<br> -<b>int pcre16_copy_named_substring(const pcre16 *<i>code</i>,</b> -<b> PCRE_SPTR16 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, PCRE_SPTR16 <i>stringname</i>,</b> -<b> PCRE_UCHAR16 *<i>buffer</i>, int <i>buffersize</i>);</b> -<br> -<br> -<b>int pcre32_copy_named_substring(const pcre32 *<i>code</i>,</b> -<b> PCRE_SPTR32 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, PCRE_SPTR32 <i>stringname</i>,</b> -<b> PCRE_UCHAR32 *<i>buffer</i>, int <i>buffersize</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This is a convenience function for extracting a captured substring, identified -by name, into a given buffer. The arguments are: -<pre> - <i>code</i> Pattern that was successfully matched - <i>subject</i> Subject that has been successfully matched - <i>ovector</i> Offset vector that <b>pcre[16|32]_exec()</b> used - <i>stringcount</i> Value returned by <b>pcre[16|32]_exec()</b> - <i>stringname</i> Name of the required substring - <i>buffer</i> Buffer to receive the string - <i>buffersize</i> Size of buffer -</pre> -The yield is the length of the substring, PCRE_ERROR_NOMEMORY if the buffer was -too small, or PCRE_ERROR_NOSUBSTRING if the string name is invalid. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_copy_substring.html b/plugins/Pcre16/docs/doc/html/pcre_copy_substring.html deleted file mode 100644 index ecaebe8533..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_copy_substring.html +++ /dev/null @@ -1,61 +0,0 @@ -<html> -<head> -<title>pcre_copy_substring specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_copy_substring man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_copy_substring(const char *<i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>, char *<i>buffer</i>,</b> -<b> int <i>buffersize</i>);</b> -<br> -<br> -<b>int pcre16_copy_substring(PCRE_SPTR16 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>, PCRE_UCHAR16 *<i>buffer</i>,</b> -<b> int <i>buffersize</i>);</b> -<br> -<br> -<b>int pcre32_copy_substring(PCRE_SPTR32 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>, PCRE_UCHAR32 *<i>buffer</i>,</b> -<b> int <i>buffersize</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This is a convenience function for extracting a captured substring into a given -buffer. The arguments are: -<pre> - <i>subject</i> Subject that has been successfully matched - <i>ovector</i> Offset vector that <b>pcre[16|32]_exec()</b> used - <i>stringcount</i> Value returned by <b>pcre[16|32]_exec()</b> - <i>stringnumber</i> Number of the required substring - <i>buffer</i> Buffer to receive the string - <i>buffersize</i> Size of buffer -</pre> -The yield is the length of the string, PCRE_ERROR_NOMEMORY if the buffer was -too small, or PCRE_ERROR_NOSUBSTRING if the string number is invalid. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_dfa_exec.html b/plugins/Pcre16/docs/doc/html/pcre_dfa_exec.html deleted file mode 100644 index 5fff6a7e0a..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_dfa_exec.html +++ /dev/null @@ -1,129 +0,0 @@ -<html> -<head> -<title>pcre_dfa_exec specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_dfa_exec man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_dfa_exec(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> -<b> const char *<i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> -<b> int *<i>workspace</i>, int <i>wscount</i>);</b> -<br> -<br> -<b>int pcre16_dfa_exec(const pcre16 *<i>code</i>, const pcre16_extra *<i>extra</i>,</b> -<b> PCRE_SPTR16 <i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> -<b> int *<i>workspace</i>, int <i>wscount</i>);</b> -<br> -<br> -<b>int pcre32_dfa_exec(const pcre32 *<i>code</i>, const pcre32_extra *<i>extra</i>,</b> -<b> PCRE_SPTR32 <i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> -<b> int *<i>workspace</i>, int <i>wscount</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function matches a compiled regular expression against a given subject -string, using an alternative matching algorithm that scans the subject string -just once (<i>not</i> Perl-compatible). Note that the main, Perl-compatible, -matching function is <b>pcre[16|32]_exec()</b>. The arguments for this function -are: -<pre> - <i>code</i> Points to the compiled pattern - <i>extra</i> Points to an associated <b>pcre[16|32]_extra</b> structure, - or is NULL - <i>subject</i> Points to the subject string - <i>length</i> Length of the subject string - <i>startoffset</i> Offset in the subject at which to start matching - <i>options</i> Option bits - <i>ovector</i> Points to a vector of ints for result offsets - <i>ovecsize</i> Number of elements in the vector - <i>workspace</i> Points to a vector of ints used as working space - <i>wscount</i> Number of elements in the vector -</pre> -The units for <i>length</i> and <i>startoffset</i> are bytes for -<b>pcre_exec()</b>, 16-bit data items for <b>pcre16_exec()</b>, and 32-bit items -for <b>pcre32_exec()</b>. The options are: -<pre> - PCRE_ANCHORED Match only at the first position - PCRE_BSR_ANYCRLF \R matches only CR, LF, or CRLF - PCRE_BSR_UNICODE \R matches all Unicode line endings - PCRE_NEWLINE_ANY Recognize any Unicode newline sequence - PCRE_NEWLINE_ANYCRLF Recognize CR, LF, & CRLF as newline sequences - PCRE_NEWLINE_CR Recognize CR as the only newline sequence - PCRE_NEWLINE_CRLF Recognize CRLF as the only newline sequence - PCRE_NEWLINE_LF Recognize LF as the only newline sequence - PCRE_NOTBOL Subject is not the beginning of a line - PCRE_NOTEOL Subject is not the end of a line - PCRE_NOTEMPTY An empty string is not a valid match - PCRE_NOTEMPTY_ATSTART An empty string at the start of the subject - is not a valid match - PCRE_NO_START_OPTIMIZE Do not do "start-match" optimizations - PCRE_NO_UTF16_CHECK Do not check the subject for UTF-16 - validity (only relevant if PCRE_UTF16 - was set at compile time) - PCRE_NO_UTF32_CHECK Do not check the subject for UTF-32 - validity (only relevant if PCRE_UTF32 - was set at compile time) - PCRE_NO_UTF8_CHECK Do not check the subject for UTF-8 - validity (only relevant if PCRE_UTF8 - was set at compile time) - PCRE_PARTIAL ) Return PCRE_ERROR_PARTIAL for a partial - PCRE_PARTIAL_SOFT ) match if no full matches are found - PCRE_PARTIAL_HARD Return PCRE_ERROR_PARTIAL for a partial match - even if there is a full match as well - PCRE_DFA_SHORTEST Return only the shortest match - PCRE_DFA_RESTART Restart after a partial match -</pre> -There are restrictions on what may appear in a pattern when using this matching -function. Details are given in the -<a href="pcrematching.html"><b>pcrematching</b></a> -documentation. For details of partial matching, see the -<a href="pcrepartial.html"><b>pcrepartial</b></a> -page. -</P> -<P> -A <b>pcre[16|32]_extra</b> structure contains the following fields: -<pre> - <i>flags</i> Bits indicating which fields are set - <i>study_data</i> Opaque data from <b>pcre[16|32]_study()</b> - <i>match_limit</i> Limit on internal resource use - <i>match_limit_recursion</i> Limit on internal recursion depth - <i>callout_data</i> Opaque data passed back to callouts - <i>tables</i> Points to character tables or is NULL - <i>mark</i> For passing back a *MARK pointer - <i>executable_jit</i> Opaque data from JIT compilation -</pre> -The flag bits are PCRE_EXTRA_STUDY_DATA, PCRE_EXTRA_MATCH_LIMIT, -PCRE_EXTRA_MATCH_LIMIT_RECURSION, PCRE_EXTRA_CALLOUT_DATA, -PCRE_EXTRA_TABLES, PCRE_EXTRA_MARK and PCRE_EXTRA_EXECUTABLE_JIT. For this -matching function, the <i>match_limit</i> and <i>match_limit_recursion</i> fields -are not used, and must not be set. The PCRE_EXTRA_EXECUTABLE_JIT flag and -the corresponding variable are ignored. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_exec.html b/plugins/Pcre16/docs/doc/html/pcre_exec.html deleted file mode 100644 index 18e1a13ff8..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_exec.html +++ /dev/null @@ -1,111 +0,0 @@ -<html> -<head> -<title>pcre_exec specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_exec man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_exec(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> -<b> const char *<i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>);</b> -<br> -<br> -<b>int pcre16_exec(const pcre16 *<i>code</i>, const pcre16_extra *<i>extra</i>,</b> -<b> PCRE_SPTR16 <i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>);</b> -<br> -<br> -<b>int pcre32_exec(const pcre32 *<i>code</i>, const pcre32_extra *<i>extra</i>,</b> -<b> PCRE_SPTR32 <i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function matches a compiled regular expression against a given subject -string, using a matching algorithm that is similar to Perl's. It returns -offsets to captured substrings. Its arguments are: -<pre> - <i>code</i> Points to the compiled pattern - <i>extra</i> Points to an associated <b>pcre[16|32]_extra</b> structure, - or is NULL - <i>subject</i> Points to the subject string - <i>length</i> Length of the subject string - <i>startoffset</i> Offset in the subject at which to start matching - <i>options</i> Option bits - <i>ovector</i> Points to a vector of ints for result offsets - <i>ovecsize</i> Number of elements in the vector (a multiple of 3) -</pre> -The units for <i>length</i> and <i>startoffset</i> are bytes for -<b>pcre_exec()</b>, 16-bit data items for <b>pcre16_exec()</b>, and 32-bit items -for <b>pcre32_exec()</b>. The options are: -<pre> - PCRE_ANCHORED Match only at the first position - PCRE_BSR_ANYCRLF \R matches only CR, LF, or CRLF - PCRE_BSR_UNICODE \R matches all Unicode line endings - PCRE_NEWLINE_ANY Recognize any Unicode newline sequence - PCRE_NEWLINE_ANYCRLF Recognize CR, LF, & CRLF as newline sequences - PCRE_NEWLINE_CR Recognize CR as the only newline sequence - PCRE_NEWLINE_CRLF Recognize CRLF as the only newline sequence - PCRE_NEWLINE_LF Recognize LF as the only newline sequence - PCRE_NOTBOL Subject string is not the beginning of a line - PCRE_NOTEOL Subject string is not the end of a line - PCRE_NOTEMPTY An empty string is not a valid match - PCRE_NOTEMPTY_ATSTART An empty string at the start of the subject - is not a valid match - PCRE_NO_START_OPTIMIZE Do not do "start-match" optimizations - PCRE_NO_UTF16_CHECK Do not check the subject for UTF-16 - validity (only relevant if PCRE_UTF16 - was set at compile time) - PCRE_NO_UTF32_CHECK Do not check the subject for UTF-32 - validity (only relevant if PCRE_UTF32 - was set at compile time) - PCRE_NO_UTF8_CHECK Do not check the subject for UTF-8 - validity (only relevant if PCRE_UTF8 - was set at compile time) - PCRE_PARTIAL ) Return PCRE_ERROR_PARTIAL for a partial - PCRE_PARTIAL_SOFT ) match if no full matches are found - PCRE_PARTIAL_HARD Return PCRE_ERROR_PARTIAL for a partial match - if that is found before a full match -</pre> -For details of partial matching, see the -<a href="pcrepartial.html"><b>pcrepartial</b></a> -page. A <b>pcre_extra</b> structure contains the following fields: -<pre> - <i>flags</i> Bits indicating which fields are set - <i>study_data</i> Opaque data from <b>pcre[16|32]_study()</b> - <i>match_limit</i> Limit on internal resource use - <i>match_limit_recursion</i> Limit on internal recursion depth - <i>callout_data</i> Opaque data passed back to callouts - <i>tables</i> Points to character tables or is NULL - <i>mark</i> For passing back a *MARK pointer - <i>executable_jit</i> Opaque data from JIT compilation -</pre> -The flag bits are PCRE_EXTRA_STUDY_DATA, PCRE_EXTRA_MATCH_LIMIT, -PCRE_EXTRA_MATCH_LIMIT_RECURSION, PCRE_EXTRA_CALLOUT_DATA, -PCRE_EXTRA_TABLES, PCRE_EXTRA_MARK and PCRE_EXTRA_EXECUTABLE_JIT. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_free_study.html b/plugins/Pcre16/docs/doc/html/pcre_free_study.html deleted file mode 100644 index 7f9e10e863..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_free_study.html +++ /dev/null @@ -1,46 +0,0 @@ -<html> -<head> -<title>pcre_free_study specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_free_study man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>void pcre_free_study(pcre_extra *<i>extra</i>);</b> -</P> -<P> -<b>void pcre16_free_study(pcre16_extra *<i>extra</i>);</b> -</P> -<P> -<b>void pcre32_free_study(pcre32_extra *<i>extra</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function is used to free the memory used for the data generated by a call -to <b>pcre[16|32]_study()</b> when it is no longer needed. The argument must be the -result of such a call. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_free_substring.html b/plugins/Pcre16/docs/doc/html/pcre_free_substring.html deleted file mode 100644 index 1fe6610746..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_free_substring.html +++ /dev/null @@ -1,46 +0,0 @@ -<html> -<head> -<title>pcre_free_substring specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_free_substring man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>void pcre_free_substring(const char *<i>stringptr</i>);</b> -</P> -<P> -<b>void pcre16_free_substring(PCRE_SPTR16 <i>stringptr</i>);</b> -</P> -<P> -<b>void pcre32_free_substring(PCRE_SPTR32 <i>stringptr</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This is a convenience function for freeing the store obtained by a previous -call to <b>pcre[16|32]_get_substring()</b> or <b>pcre[16|32]_get_named_substring()</b>. -Its only argument is a pointer to the string. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_free_substring_list.html b/plugins/Pcre16/docs/doc/html/pcre_free_substring_list.html deleted file mode 100644 index c0861780b4..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_free_substring_list.html +++ /dev/null @@ -1,46 +0,0 @@ -<html> -<head> -<title>pcre_free_substring_list specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_free_substring_list man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>void pcre_free_substring_list(const char **<i>stringptr</i>);</b> -</P> -<P> -<b>void pcre16_free_substring_list(PCRE_SPTR16 *<i>stringptr</i>);</b> -</P> -<P> -<b>void pcre32_free_substring_list(PCRE_SPTR32 *<i>stringptr</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This is a convenience function for freeing the store obtained by a previous -call to <b>pcre[16|32]_get_substring_list()</b>. Its only argument is a pointer to -the list of string pointers. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_fullinfo.html b/plugins/Pcre16/docs/doc/html/pcre_fullinfo.html deleted file mode 100644 index b88fc1155b..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_fullinfo.html +++ /dev/null @@ -1,108 +0,0 @@ -<html> -<head> -<title>pcre_fullinfo specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_fullinfo man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_fullinfo(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> -<b> int <i>what</i>, void *<i>where</i>);</b> -<br> -<br> -<b>int pcre16_fullinfo(const pcre16 *<i>code</i>, const pcre16_extra *<i>extra</i>,</b> -<b> int <i>what</i>, void *<i>where</i>);</b> -<br> -<br> -<b>int pcre32_fullinfo(const pcre32 *<i>code</i>, const pcre32_extra *<i>extra</i>,</b> -<b> int <i>what</i>, void *<i>where</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function returns information about a compiled pattern. Its arguments are: -<pre> - <i>code</i> Compiled regular expression - <i>extra</i> Result of <b>pcre[16|32]_study()</b> or NULL - <i>what</i> What information is required - <i>where</i> Where to put the information -</pre> -The following information is available: -<pre> - PCRE_INFO_BACKREFMAX Number of highest back reference - PCRE_INFO_CAPTURECOUNT Number of capturing subpatterns - PCRE_INFO_DEFAULT_TABLES Pointer to default tables - PCRE_INFO_FIRSTBYTE Fixed first data unit for a match, or - -1 for start of string - or after newline, or - -2 otherwise - PCRE_INFO_FIRSTTABLE Table of first data units (after studying) - PCRE_INFO_HASCRORLF Return 1 if explicit CR or LF matches exist - PCRE_INFO_JCHANGED Return 1 if (?J) or (?-J) was used - PCRE_INFO_JIT Return 1 after successful JIT compilation - PCRE_INFO_JITSIZE Size of JIT compiled code - PCRE_INFO_LASTLITERAL Literal last data unit required - PCRE_INFO_MINLENGTH Lower bound length of matching strings - PCRE_INFO_NAMECOUNT Number of named subpatterns - PCRE_INFO_NAMEENTRYSIZE Size of name table entry - PCRE_INFO_NAMETABLE Pointer to name table - PCRE_INFO_OKPARTIAL Return 1 if partial matching can be tried - (always returns 1 after release 8.00) - PCRE_INFO_OPTIONS Option bits used for compilation - PCRE_INFO_SIZE Size of compiled pattern - PCRE_INFO_STUDYSIZE Size of study data - PCRE_INFO_FIRSTCHARACTER Fixed first data unit for a match - PCRE_INFO_FIRSTCHARACTERFLAGS Returns - 1 if there is a first data character set, which can - then be retrieved using PCRE_INFO_FIRSTCHARACTER, - 2 if the first character is at the start of the data - string or after a newline, and - 0 otherwise - PCRE_INFO_REQUIREDCHAR Literal last data unit required - PCRE_INFO_REQUIREDCHARFLAGS Returns 1 if the last data character is set (which can then - be retrieved using PCRE_INFO_REQUIREDCHAR); 0 otherwise -</pre> -The <i>where</i> argument must point to an integer variable, except for the -following <i>what</i> values: -<pre> - PCRE_INFO_DEFAULT_TABLES const unsigned char * - PCRE_INFO_FIRSTTABLE const unsigned char * - PCRE_INFO_NAMETABLE PCRE_SPTR16 (16-bit library) - PCRE_INFO_NAMETABLE PCRE_SPTR32 (32-bit library) - PCRE_INFO_NAMETABLE const unsigned char * (8-bit library) - PCRE_INFO_OPTIONS unsigned long int - PCRE_INFO_SIZE size_t - PCRE_INFO_FIRSTCHARACTER uint32_t - PCRE_INFO_REQUIREDCHAR uint32_t -</pre> -The yield of the function is zero on success or: -<pre> - PCRE_ERROR_NULL the argument <i>code</i> was NULL - the argument <i>where</i> was NULL - PCRE_ERROR_BADMAGIC the "magic number" was not found - PCRE_ERROR_BADOPTION the value of <i>what</i> was invalid -</PRE> -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_get_named_substring.html b/plugins/Pcre16/docs/doc/html/pcre_get_named_substring.html deleted file mode 100644 index 72924d9b25..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_get_named_substring.html +++ /dev/null @@ -1,68 +0,0 @@ -<html> -<head> -<title>pcre_get_named_substring specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_get_named_substring man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_get_named_substring(const pcre *<i>code</i>,</b> -<b> const char *<i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, const char *<i>stringname</i>,</b> -<b> const char **<i>stringptr</i>);</b> -<br> -<br> -<b>int pcre16_get_named_substring(const pcre16 *<i>code</i>,</b> -<b> PCRE_SPTR16 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, PCRE_SPTR16 <i>stringname</i>,</b> -<b> PCRE_SPTR16 *<i>stringptr</i>);</b> -<br> -<br> -<b>int pcre32_get_named_substring(const pcre32 *<i>code</i>,</b> -<b> PCRE_SPTR32 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, PCRE_SPTR32 <i>stringname</i>,</b> -<b> PCRE_SPTR32 *<i>stringptr</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This is a convenience function for extracting a captured substring by name. The -arguments are: -<pre> - <i>code</i> Compiled pattern - <i>subject</i> Subject that has been successfully matched - <i>ovector</i> Offset vector that <b>pcre[16|32]_exec()</b> used - <i>stringcount</i> Value returned by <b>pcre[16|32]_exec()</b> - <i>stringname</i> Name of the required substring - <i>stringptr</i> Where to put the string pointer -</pre> -The memory in which the substring is placed is obtained by calling -<b>pcre[16|32]_malloc()</b>. The convenience function -<b>pcre[16|32]_free_substring()</b> can be used to free it when it is no longer -needed. The yield of the function is the length of the extracted substring, -PCRE_ERROR_NOMEMORY if sufficient memory could not be obtained, or -PCRE_ERROR_NOSUBSTRING if the string name is invalid. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_get_stringnumber.html b/plugins/Pcre16/docs/doc/html/pcre_get_stringnumber.html deleted file mode 100644 index 7324d782e7..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_get_stringnumber.html +++ /dev/null @@ -1,57 +0,0 @@ -<html> -<head> -<title>pcre_get_stringnumber specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_get_stringnumber man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_get_stringnumber(const pcre *<i>code</i>,</b> -<b> const char *<i>name</i>);</b> -<br> -<br> -<b>int pcre16_get_stringnumber(const pcre16 *<i>code</i>,</b> -<b> PCRE_SPTR16 <i>name</i>);</b> -<br> -<br> -<b>int pcre32_get_stringnumber(const pcre32 *<i>code</i>,</b> -<b> PCRE_SPTR32 <i>name</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This convenience function finds the number of a named substring capturing -parenthesis in a compiled pattern. Its arguments are: -<pre> - <i>code</i> Compiled regular expression - <i>name</i> Name whose number is required -</pre> -The yield of the function is the number of the parenthesis if the name is -found, or PCRE_ERROR_NOSUBSTRING otherwise. When duplicate names are allowed -(PCRE_DUPNAMES is set), it is not defined which of the numbers is returned by -<b>pcre[16|32]_get_stringnumber()</b>. You can obtain the complete list by calling -<b>pcre[16|32]_get_stringtable_entries()</b>. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_get_stringtable_entries.html b/plugins/Pcre16/docs/doc/html/pcre_get_stringtable_entries.html deleted file mode 100644 index 79906798e6..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_get_stringtable_entries.html +++ /dev/null @@ -1,60 +0,0 @@ -<html> -<head> -<title>pcre_get_stringtable_entries specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_get_stringtable_entries man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_get_stringtable_entries(const pcre *<i>code</i>,</b> -<b> const char *<i>name</i>, char **<i>first</i>, char **<i>last</i>);</b> -<br> -<br> -<b>int pcre16_get_stringtable_entries(const pcre16 *<i>code</i>,</b> -<b> PCRE_SPTR16 <i>name</i>, PCRE_UCHAR16 **<i>first</i>, PCRE_UCHAR16 **<i>last</i>);</b> -<br> -<br> -<b>int pcre32_get_stringtable_entries(const pcre32 *<i>code</i>,</b> -<b> PCRE_SPTR32 <i>name</i>, PCRE_UCHAR32 **<i>first</i>, PCRE_UCHAR32 **<i>last</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This convenience function finds, for a compiled pattern, the first and last -entries for a given name in the table that translates capturing parenthesis -names into numbers. When names are required to be unique (PCRE_DUPNAMES is -<i>not</i> set), it is usually easier to use <b>pcre[16|32]_get_stringnumber()</b> -instead. -<pre> - <i>code</i> Compiled regular expression - <i>name</i> Name whose entries required - <i>first</i> Where to return a pointer to the first entry - <i>last</i> Where to return a pointer to the last entry -</pre> -The yield of the function is the length of each entry, or -PCRE_ERROR_NOSUBSTRING if none are found. -</P> -<P> -There is a complete description of the PCRE native API, including the format of -the table entries, in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page, and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_get_substring.html b/plugins/Pcre16/docs/doc/html/pcre_get_substring.html deleted file mode 100644 index 1a8e4f5a49..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_get_substring.html +++ /dev/null @@ -1,64 +0,0 @@ -<html> -<head> -<title>pcre_get_substring specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_get_substring man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_get_substring(const char *<i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>,</b> -<b> const char **<i>stringptr</i>);</b> -<br> -<br> -<b>int pcre16_get_substring(PCRE_SPTR16 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>,</b> -<b> PCRE_SPTR16 *<i>stringptr</i>);</b> -<br> -<br> -<b>int pcre32_get_substring(PCRE_SPTR32 <i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>,</b> -<b> PCRE_SPTR32 *<i>stringptr</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This is a convenience function for extracting a captured substring. The -arguments are: -<pre> - <i>subject</i> Subject that has been successfully matched - <i>ovector</i> Offset vector that <b>pcre[16|32]_exec()</b> used - <i>stringcount</i> Value returned by <b>pcre[16|32]_exec()</b> - <i>stringnumber</i> Number of the required substring - <i>stringptr</i> Where to put the string pointer -</pre> -The memory in which the substring is placed is obtained by calling -<b>pcre[16|32]_malloc()</b>. The convenience function -<b>pcre[16|32]_free_substring()</b> can be used to free it when it is no longer -needed. The yield of the function is the length of the substring, -PCRE_ERROR_NOMEMORY if sufficient memory could not be obtained, or -PCRE_ERROR_NOSUBSTRING if the string number is invalid. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_get_substring_list.html b/plugins/Pcre16/docs/doc/html/pcre_get_substring_list.html deleted file mode 100644 index 7e8c6bc858..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_get_substring_list.html +++ /dev/null @@ -1,61 +0,0 @@ -<html> -<head> -<title>pcre_get_substring_list specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_get_substring_list man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_get_substring_list(const char *<i>subject</i>,</b> -<b> int *<i>ovector</i>, int <i>stringcount</i>, const char ***<i>listptr</i>);</b> -<br> -<br> -<b>int pcre16_get_substring_list(PCRE_SPTR16 <i>subject</i>,</b> -<b> int *<i>ovector</i>, int <i>stringcount</i>, PCRE_SPTR16 **<i>listptr</i>);</b> -<br> -<br> -<b>int pcre32_get_substring_list(PCRE_SPTR32 <i>subject</i>,</b> -<b> int *<i>ovector</i>, int <i>stringcount</i>, PCRE_SPTR32 **<i>listptr</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This is a convenience function for extracting a list of all the captured -substrings. The arguments are: -<pre> - <i>subject</i> Subject that has been successfully matched - <i>ovector</i> Offset vector that <b>pcre[16|32]_exec</b> used - <i>stringcount</i> Value returned by <b>pcre[16|32]_exec</b> - <i>listptr</i> Where to put a pointer to the list -</pre> -The memory in which the substrings and the list are placed is obtained by -calling <b>pcre[16|32]_malloc()</b>. The convenience function -<b>pcre[16|32]_free_substring_list()</b> can be used to free it when it is no -longer needed. A pointer to a list of pointers is put in the variable whose -address is in <i>listptr</i>. The list is terminated by a NULL pointer. The -yield of the function is zero on success or PCRE_ERROR_NOMEMORY if sufficient -memory could not be obtained. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_jit_exec.html b/plugins/Pcre16/docs/doc/html/pcre_jit_exec.html deleted file mode 100644 index 4ebb0cbcac..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_jit_exec.html +++ /dev/null @@ -1,108 +0,0 @@ -<html> -<head> -<title>pcre_jit_exec specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_jit_exec man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_jit_exec(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> -<b> const char *<i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> -<b> pcre_jit_stack *<i>jstack</i>);</b> -<br> -<br> -<b>int pcre16_jit_exec(const pcre16 *<i>code</i>, const pcre16_extra *<i>extra</i>,</b> -<b> PCRE_SPTR16 <i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> -<b> pcre_jit_stack *<i>jstack</i>);</b> -<br> -<br> -<b>int pcre32_jit_exec(const pcre32 *<i>code</i>, const pcre32_extra *<i>extra</i>,</b> -<b> PCRE_SPTR32 <i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> -<b> pcre_jit_stack *<i>jstack</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function matches a compiled regular expression that has been successfully -studied with one of the JIT options against a given subject string, using a -matching algorithm that is similar to Perl's. It is a "fast path" interface to -JIT, and it bypasses some of the sanity checks that <b>pcre_exec()</b> applies. -It returns offsets to captured substrings. Its arguments are: -<pre> - <i>code</i> Points to the compiled pattern - <i>extra</i> Points to an associated <b>pcre[16|32]_extra</b> structure, - or is NULL - <i>subject</i> Points to the subject string - <i>length</i> Length of the subject string, in bytes - <i>startoffset</i> Offset in bytes in the subject at which to - start matching - <i>options</i> Option bits - <i>ovector</i> Points to a vector of ints for result offsets - <i>ovecsize</i> Number of elements in the vector (a multiple of 3) - <i>jstack</i> Pointer to a JIT stack -</pre> -The allowed options are: -<pre> - PCRE_NOTBOL Subject string is not the beginning of a line - PCRE_NOTEOL Subject string is not the end of a line - PCRE_NOTEMPTY An empty string is not a valid match - PCRE_NOTEMPTY_ATSTART An empty string at the start of the subject - is not a valid match - PCRE_NO_UTF16_CHECK Do not check the subject for UTF-16 - validity (only relevant if PCRE_UTF16 - was set at compile time) - PCRE_NO_UTF32_CHECK Do not check the subject for UTF-32 - validity (only relevant if PCRE_UTF32 - was set at compile time) - PCRE_NO_UTF8_CHECK Do not check the subject for UTF-8 - validity (only relevant if PCRE_UTF8 - was set at compile time) - PCRE_PARTIAL ) Return PCRE_ERROR_PARTIAL for a partial - PCRE_PARTIAL_SOFT ) match if no full matches are found - PCRE_PARTIAL_HARD Return PCRE_ERROR_PARTIAL for a partial match - if that is found before a full match -</pre> -However, the PCRE_NO_UTF[8|16|32]_CHECK options have no effect, as this check -is never applied. For details of partial matching, see the -<a href="pcrepartial.html"><b>pcrepartial</b></a> -page. A <b>pcre_extra</b> structure contains the following fields: -<pre> - <i>flags</i> Bits indicating which fields are set - <i>study_data</i> Opaque data from <b>pcre[16|32]_study()</b> - <i>match_limit</i> Limit on internal resource use - <i>match_limit_recursion</i> Limit on internal recursion depth - <i>callout_data</i> Opaque data passed back to callouts - <i>tables</i> Points to character tables or is NULL - <i>mark</i> For passing back a *MARK pointer - <i>executable_jit</i> Opaque data from JIT compilation -</pre> -The flag bits are PCRE_EXTRA_STUDY_DATA, PCRE_EXTRA_MATCH_LIMIT, -PCRE_EXTRA_MATCH_LIMIT_RECURSION, PCRE_EXTRA_CALLOUT_DATA, -PCRE_EXTRA_TABLES, PCRE_EXTRA_MARK and PCRE_EXTRA_EXECUTABLE_JIT. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the JIT API in the -<a href="pcrejit.html"><b>pcrejit</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_jit_stack_alloc.html b/plugins/Pcre16/docs/doc/html/pcre_jit_stack_alloc.html deleted file mode 100644 index 23ba450750..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_jit_stack_alloc.html +++ /dev/null @@ -1,55 +0,0 @@ -<html> -<head> -<title>pcre_jit_stack_alloc specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_jit_stack_alloc man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>pcre_jit_stack *pcre_jit_stack_alloc(int <i>startsize</i>,</b> -<b> int <i>maxsize</i>);</b> -<br> -<br> -<b>pcre16_jit_stack *pcre16_jit_stack_alloc(int <i>startsize</i>,</b> -<b> int <i>maxsize</i>);</b> -<br> -<br> -<b>pcre32_jit_stack *pcre32_jit_stack_alloc(int <i>startsize</i>,</b> -<b> int <i>maxsize</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function is used to create a stack for use by the code compiled by the JIT -optimization of <b>pcre[16|32]_study()</b>. The arguments are a starting size for -the stack, and a maximum size to which it is allowed to grow. The result can be -passed to the JIT run-time code by <b>pcre[16|32]_assign_jit_stack()</b>, or that -function can set up a callback for obtaining a stack. A maximum stack size of -512K to 1M should be more than enough for any pattern. For more details, see -the -<a href="pcrejit.html"><b>pcrejit</b></a> -page. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_jit_stack_free.html b/plugins/Pcre16/docs/doc/html/pcre_jit_stack_free.html deleted file mode 100644 index 8bd06e4655..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_jit_stack_free.html +++ /dev/null @@ -1,48 +0,0 @@ -<html> -<head> -<title>pcre_jit_stack_free specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_jit_stack_free man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>void pcre_jit_stack_free(pcre_jit_stack *<i>stack</i>);</b> -</P> -<P> -<b>void pcre16_jit_stack_free(pcre16_jit_stack *<i>stack</i>);</b> -</P> -<P> -<b>void pcre32_jit_stack_free(pcre32_jit_stack *<i>stack</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function is used to free a JIT stack that was created by -<b>pcre[16|32]_jit_stack_alloc()</b> when it is no longer needed. For more details, -see the -<a href="pcrejit.html"><b>pcrejit</b></a> -page. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_maketables.html b/plugins/Pcre16/docs/doc/html/pcre_maketables.html deleted file mode 100644 index 3a7b5ebc4a..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_maketables.html +++ /dev/null @@ -1,48 +0,0 @@ -<html> -<head> -<title>pcre_maketables specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_maketables man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>const unsigned char *pcre_maketables(void);</b> -</P> -<P> -<b>const unsigned char *pcre16_maketables(void);</b> -</P> -<P> -<b>const unsigned char *pcre32_maketables(void);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function builds a set of character tables for character values less than -256. These can be passed to <b>pcre[16|32]_compile()</b> to override PCRE's -internal, built-in tables (which were made by <b>pcre[16|32]_maketables()</b> when -PCRE was compiled). You might want to do this if you are using a non-standard -locale. The function yields a pointer to the tables. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_pattern_to_host_byte_order.html b/plugins/Pcre16/docs/doc/html/pcre_pattern_to_host_byte_order.html deleted file mode 100644 index 1b1c80372b..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_pattern_to_host_byte_order.html +++ /dev/null @@ -1,58 +0,0 @@ -<html> -<head> -<title>pcre_pattern_to_host_byte_order specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_pattern_to_host_byte_order man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_pattern_to_host_byte_order(pcre *<i>code</i>,</b> -<b> pcre_extra *<i>extra</i>, const unsigned char *<i>tables</i>);</b> -<br> -<br> -<b>int pcre16_pattern_to_host_byte_order(pcre16 *<i>code</i>,</b> -<b> pcre16_extra *<i>extra</i>, const unsigned char *<i>tables</i>);</b> -<br> -<br> -<b>int pcre32_pattern_to_host_byte_order(pcre32 *<i>code</i>,</b> -<b> pcre32_extra *<i>extra</i>, const unsigned char *<i>tables</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function ensures that the bytes in 2-byte and 4-byte values in a compiled -pattern are in the correct order for the current host. It is useful when a -pattern that has been compiled on one host is transferred to another that might -have different endianness. The arguments are: -<pre> - <i>code</i> A compiled regular expression - <i>extra</i> Points to an associated <b>pcre[16|32]_extra</b> structure, - or is NULL - <i>tables</i> Pointer to character tables, or NULL to - set the built-in default -</pre> -The result is 0 for success, a negative PCRE_ERROR_xxx value otherwise. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_refcount.html b/plugins/Pcre16/docs/doc/html/pcre_refcount.html deleted file mode 100644 index bfb92e6d8a..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_refcount.html +++ /dev/null @@ -1,51 +0,0 @@ -<html> -<head> -<title>pcre_refcount specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_refcount man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre_refcount(pcre *<i>code</i>, int <i>adjust</i>);</b> -</P> -<P> -<b>int pcre16_refcount(pcre16 *<i>code</i>, int <i>adjust</i>);</b> -</P> -<P> -<b>int pcre32_refcount(pcre32 *<i>code</i>, int <i>adjust</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function is used to maintain a reference count inside a data block that -contains a compiled pattern. Its arguments are: -<pre> - <i>code</i> Compiled regular expression - <i>adjust</i> Adjustment to reference value -</pre> -The yield of the function is the adjusted reference value, which is constrained -to lie between 0 and 65535. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_study.html b/plugins/Pcre16/docs/doc/html/pcre_study.html deleted file mode 100644 index af82f11409..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_study.html +++ /dev/null @@ -1,68 +0,0 @@ -<html> -<head> -<title>pcre_study specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_study man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>pcre_extra *pcre_study(const pcre *<i>code</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>);</b> -<br> -<br> -<b>pcre16_extra *pcre16_study(const pcre16 *<i>code</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>);</b> -<br> -<br> -<b>pcre32_extra *pcre32_study(const pcre32 *<i>code</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function studies a compiled pattern, to see if additional information can -be extracted that might speed up matching. Its arguments are: -<pre> - <i>code</i> A compiled regular expression - <i>options</i> Options for <b>pcre[16|32]_study()</b> - <i>errptr</i> Where to put an error message -</pre> -If the function succeeds, it returns a value that can be passed to -<b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> via their <i>extra</i> -arguments. -</P> -<P> -If the function returns NULL, either it could not find any additional -information, or there was an error. You can tell the difference by looking at -the error value. It is NULL in first case. -</P> -<P> -The only option is PCRE_STUDY_JIT_COMPILE. It requests just-in-time compilation -if possible. If PCRE has been compiled without JIT support, this option is -ignored. See the -<a href="pcrejit.html"><b>pcrejit</b></a> -page for further details. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_utf16_to_host_byte_order.html b/plugins/Pcre16/docs/doc/html/pcre_utf16_to_host_byte_order.html deleted file mode 100644 index 18e7788f68..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_utf16_to_host_byte_order.html +++ /dev/null @@ -1,57 +0,0 @@ -<html> -<head> -<title>pcre_utf16_to_host_byte_order specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_utf16_to_host_byte_order man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre16_utf16_to_host_byte_order(PCRE_UCHAR16 *<i>output</i>,</b> -<b> PCRE_SPTR16 <i>input</i>, int <i>length</i>, int *<i>host_byte_order</i>,</b> -<b> int <i>keep_boms</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function, which exists only in the 16-bit library, converts a UTF-16 -string to the correct order for the current host, taking account of any byte -order marks (BOMs) within the string. Its arguments are: -<pre> - <i>output</i> pointer to output buffer, may be the same as <i>input</i> - <i>input</i> pointer to input buffer - <i>length</i> number of 16-bit units in the input, or negative for - a zero-terminated string - <i>host_byte_order</i> a NULL value or a non-zero value pointed to means - start in host byte order - <i>keep_boms</i> if non-zero, BOMs are copied to the output string -</pre> -The result of the function is the number of 16-bit units placed into the output -buffer, including the zero terminator if the string was zero-terminated. -</P> -<P> -If <i>host_byte_order</i> is not NULL, it is set to indicate the byte order that -is current at the end of the string. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_utf32_to_host_byte_order.html b/plugins/Pcre16/docs/doc/html/pcre_utf32_to_host_byte_order.html deleted file mode 100644 index 772ae40cd9..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_utf32_to_host_byte_order.html +++ /dev/null @@ -1,57 +0,0 @@ -<html> -<head> -<title>pcre_utf32_to_host_byte_order specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_utf32_to_host_byte_order man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int pcre32_utf32_to_host_byte_order(PCRE_UCHAR32 *<i>output</i>,</b> -<b> PCRE_SPTR32 <i>input</i>, int <i>length</i>, int *<i>host_byte_order</i>,</b> -<b> int <i>keep_boms</i>);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function, which exists only in the 32-bit library, converts a UTF-32 -string to the correct order for the current host, taking account of any byte -order marks (BOMs) within the string. Its arguments are: -<pre> - <i>output</i> pointer to output buffer, may be the same as <i>input</i> - <i>input</i> pointer to input buffer - <i>length</i> number of 32-bit units in the input, or negative for - a zero-terminated string - <i>host_byte_order</i> a NULL value or a non-zero value pointed to means - start in host byte order - <i>keep_boms</i> if non-zero, BOMs are copied to the output string -</pre> -The result of the function is the number of 32-bit units placed into the output -buffer, including the zero terminator if the string was zero-terminated. -</P> -<P> -If <i>host_byte_order</i> is not NULL, it is set to indicate the byte order that -is current at the end of the string. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcre_version.html b/plugins/Pcre16/docs/doc/html/pcre_version.html deleted file mode 100644 index d33e718955..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcre_version.html +++ /dev/null @@ -1,46 +0,0 @@ -<html> -<head> -<title>pcre_version specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcre_version man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SYNOPSIS -</b><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>const char *pcre_version(void);</b> -</P> -<P> -<b>const char *pcre16_version(void);</b> -</P> -<P> -<b>const char *pcre32_version(void);</b> -</P> -<br><b> -DESCRIPTION -</b><br> -<P> -This function (even in the 16-bit and 32-bit libraries) returns a -zero-terminated, 8-bit character string that gives the version number of the -PCRE library and the date of its release. -</P> -<P> -There is a complete description of the PCRE native API in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page and a description of the POSIX API in the -<a href="pcreposix.html"><b>pcreposix</b></a> -page. -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcreapi.html b/plugins/Pcre16/docs/doc/html/pcreapi.html deleted file mode 100644 index b401ecc76d..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcreapi.html +++ /dev/null @@ -1,2922 +0,0 @@ -<html> -<head> -<title>pcreapi specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcreapi man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">PCRE NATIVE API BASIC FUNCTIONS</a> -<li><a name="TOC2" href="#SEC2">PCRE NATIVE API STRING EXTRACTION FUNCTIONS</a> -<li><a name="TOC3" href="#SEC3">PCRE NATIVE API AUXILIARY FUNCTIONS</a> -<li><a name="TOC4" href="#SEC4">PCRE NATIVE API INDIRECTED FUNCTIONS</a> -<li><a name="TOC5" href="#SEC5">PCRE 8-BIT, 16-BIT, AND 32-BIT LIBRARIES</a> -<li><a name="TOC6" href="#SEC6">PCRE API OVERVIEW</a> -<li><a name="TOC7" href="#SEC7">NEWLINES</a> -<li><a name="TOC8" href="#SEC8">MULTITHREADING</a> -<li><a name="TOC9" href="#SEC9">SAVING PRECOMPILED PATTERNS FOR LATER USE</a> -<li><a name="TOC10" href="#SEC10">CHECKING BUILD-TIME OPTIONS</a> -<li><a name="TOC11" href="#SEC11">COMPILING A PATTERN</a> -<li><a name="TOC12" href="#SEC12">COMPILATION ERROR CODES</a> -<li><a name="TOC13" href="#SEC13">STUDYING A PATTERN</a> -<li><a name="TOC14" href="#SEC14">LOCALE SUPPORT</a> -<li><a name="TOC15" href="#SEC15">INFORMATION ABOUT A PATTERN</a> -<li><a name="TOC16" href="#SEC16">REFERENCE COUNTS</a> -<li><a name="TOC17" href="#SEC17">MATCHING A PATTERN: THE TRADITIONAL FUNCTION</a> -<li><a name="TOC18" href="#SEC18">EXTRACTING CAPTURED SUBSTRINGS BY NUMBER</a> -<li><a name="TOC19" href="#SEC19">EXTRACTING CAPTURED SUBSTRINGS BY NAME</a> -<li><a name="TOC20" href="#SEC20">DUPLICATE SUBPATTERN NAMES</a> -<li><a name="TOC21" href="#SEC21">FINDING ALL POSSIBLE MATCHES</a> -<li><a name="TOC22" href="#SEC22">OBTAINING AN ESTIMATE OF STACK USAGE</a> -<li><a name="TOC23" href="#SEC23">MATCHING A PATTERN: THE ALTERNATIVE FUNCTION</a> -<li><a name="TOC24" href="#SEC24">SEE ALSO</a> -<li><a name="TOC25" href="#SEC25">AUTHOR</a> -<li><a name="TOC26" href="#SEC26">REVISION</a> -</ul> -<P> -<b>#include <pcre.h></b> -</P> -<br><a name="SEC1" href="#TOC1">PCRE NATIVE API BASIC FUNCTIONS</a><br> -<P> -<b>pcre *pcre_compile(const char *<i>pattern</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -<br> -<br> -<b>pcre *pcre_compile2(const char *<i>pattern</i>, int <i>options</i>,</b> -<b> int *<i>errorcodeptr</i>,</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -<br> -<br> -<b>pcre_extra *pcre_study(const pcre *<i>code</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>);</b> -<br> -<br> -<b>void pcre_free_study(pcre_extra *<i>extra</i>);</b> -<br> -<br> -<b>int pcre_exec(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> -<b> const char *<i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>);</b> -<br> -<br> -<b>int pcre_dfa_exec(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> -<b> const char *<i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> -<b> int *<i>workspace</i>, int <i>wscount</i>);</b> -</P> -<br><a name="SEC2" href="#TOC1">PCRE NATIVE API STRING EXTRACTION FUNCTIONS</a><br> -<P> -<b>int pcre_copy_named_substring(const pcre *<i>code</i>,</b> -<b> const char *<i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, const char *<i>stringname</i>,</b> -<b> char *<i>buffer</i>, int <i>buffersize</i>);</b> -<br> -<br> -<b>int pcre_copy_substring(const char *<i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>, char *<i>buffer</i>,</b> -<b> int <i>buffersize</i>);</b> -<br> -<br> -<b>int pcre_get_named_substring(const pcre *<i>code</i>,</b> -<b> const char *<i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, const char *<i>stringname</i>,</b> -<b> const char **<i>stringptr</i>);</b> -<br> -<br> -<b>int pcre_get_stringnumber(const pcre *<i>code</i>,</b> -<b> const char *<i>name</i>);</b> -<br> -<br> -<b>int pcre_get_stringtable_entries(const pcre *<i>code</i>,</b> -<b> const char *<i>name</i>, char **<i>first</i>, char **<i>last</i>);</b> -<br> -<br> -<b>int pcre_get_substring(const char *<i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>,</b> -<b> const char **<i>stringptr</i>);</b> -<br> -<br> -<b>int pcre_get_substring_list(const char *<i>subject</i>,</b> -<b> int *<i>ovector</i>, int <i>stringcount</i>, const char ***<i>listptr</i>);</b> -<br> -<br> -<b>void pcre_free_substring(const char *<i>stringptr</i>);</b> -<br> -<br> -<b>void pcre_free_substring_list(const char **<i>stringptr</i>);</b> -</P> -<br><a name="SEC3" href="#TOC1">PCRE NATIVE API AUXILIARY FUNCTIONS</a><br> -<P> -<b>int pcre_jit_exec(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> -<b> const char *<i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> -<b> pcre_jit_stack *<i>jstack</i>);</b> -<br> -<br> -<b>pcre_jit_stack *pcre_jit_stack_alloc(int <i>startsize</i>, int <i>maxsize</i>);</b> -<br> -<br> -<b>void pcre_jit_stack_free(pcre_jit_stack *<i>stack</i>);</b> -<br> -<br> -<b>void pcre_assign_jit_stack(pcre_extra *<i>extra</i>,</b> -<b> pcre_jit_callback <i>callback</i>, void *<i>data</i>);</b> -<br> -<br> -<b>const unsigned char *pcre_maketables(void);</b> -<br> -<br> -<b>int pcre_fullinfo(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> -<b> int <i>what</i>, void *<i>where</i>);</b> -<br> -<br> -<b>int pcre_refcount(pcre *<i>code</i>, int <i>adjust</i>);</b> -<br> -<br> -<b>int pcre_config(int <i>what</i>, void *<i>where</i>);</b> -<br> -<br> -<b>const char *pcre_version(void);</b> -<br> -<br> -<b>int pcre_pattern_to_host_byte_order(pcre *<i>code</i>,</b> -<b> pcre_extra *<i>extra</i>, const unsigned char *<i>tables</i>);</b> -</P> -<br><a name="SEC4" href="#TOC1">PCRE NATIVE API INDIRECTED FUNCTIONS</a><br> -<P> -<b>void *(*pcre_malloc)(size_t);</b> -<br> -<br> -<b>void (*pcre_free)(void *);</b> -<br> -<br> -<b>void *(*pcre_stack_malloc)(size_t);</b> -<br> -<br> -<b>void (*pcre_stack_free)(void *);</b> -<br> -<br> -<b>int (*pcre_callout)(pcre_callout_block *);</b> -<br> -<br> -<b>int (*pcre_stack_guard)(void);</b> -</P> -<br><a name="SEC5" href="#TOC1">PCRE 8-BIT, 16-BIT, AND 32-BIT LIBRARIES</a><br> -<P> -As well as support for 8-bit character strings, PCRE also supports 16-bit -strings (from release 8.30) and 32-bit strings (from release 8.32), by means of -two additional libraries. They can be built as well as, or instead of, the -8-bit library. To avoid too much complication, this document describes the -8-bit versions of the functions, with only occasional references to the 16-bit -and 32-bit libraries. -</P> -<P> -The 16-bit and 32-bit functions operate in the same way as their 8-bit -counterparts; they just use different data types for their arguments and -results, and their names start with <b>pcre16_</b> or <b>pcre32_</b> instead of -<b>pcre_</b>. For every option that has UTF8 in its name (for example, -PCRE_UTF8), there are corresponding 16-bit and 32-bit names with UTF8 replaced -by UTF16 or UTF32, respectively. This facility is in fact just cosmetic; the -16-bit and 32-bit option names define the same bit values. -</P> -<P> -References to bytes and UTF-8 in this document should be read as references to -16-bit data units and UTF-16 when using the 16-bit library, or 32-bit data -units and UTF-32 when using the 32-bit library, unless specified otherwise. -More details of the specific differences for the 16-bit and 32-bit libraries -are given in the -<a href="pcre16.html"><b>pcre16</b></a> -and -<a href="pcre32.html"><b>pcre32</b></a> -pages. -</P> -<br><a name="SEC6" href="#TOC1">PCRE API OVERVIEW</a><br> -<P> -PCRE has its own native API, which is described in this document. There are -also some wrapper functions (for the 8-bit library only) that correspond to the -POSIX regular expression API, but they do not give access to all the -functionality. They are described in the -<a href="pcreposix.html"><b>pcreposix</b></a> -documentation. Both of these APIs define a set of C function calls. A C++ -wrapper (again for the 8-bit library only) is also distributed with PCRE. It is -documented in the -<a href="pcrecpp.html"><b>pcrecpp</b></a> -page. -</P> -<P> -The native API C function prototypes are defined in the header file -<b>pcre.h</b>, and on Unix-like systems the (8-bit) library itself is called -<b>libpcre</b>. It can normally be accessed by adding <b>-lpcre</b> to the -command for linking an application that uses PCRE. The header file defines the -macros PCRE_MAJOR and PCRE_MINOR to contain the major and minor release numbers -for the library. Applications can use these to include support for different -releases of PCRE. -</P> -<P> -In a Windows environment, if you want to statically link an application program -against a non-dll <b>pcre.a</b> file, you must define PCRE_STATIC before -including <b>pcre.h</b> or <b>pcrecpp.h</b>, because otherwise the -<b>pcre_malloc()</b> and <b>pcre_free()</b> exported functions will be declared -<b>__declspec(dllimport)</b>, with unwanted results. -</P> -<P> -The functions <b>pcre_compile()</b>, <b>pcre_compile2()</b>, <b>pcre_study()</b>, -and <b>pcre_exec()</b> are used for compiling and matching regular expressions -in a Perl-compatible manner. A sample program that demonstrates the simplest -way of using them is provided in the file called <i>pcredemo.c</i> in the PCRE -source distribution. A listing of this program is given in the -<a href="pcredemo.html"><b>pcredemo</b></a> -documentation, and the -<a href="pcresample.html"><b>pcresample</b></a> -documentation describes how to compile and run it. -</P> -<P> -Just-in-time compiler support is an optional feature of PCRE that can be built -in appropriate hardware environments. It greatly speeds up the matching -performance of many patterns. Simple programs can easily request that it be -used if available, by setting an option that is ignored when it is not -relevant. More complicated programs might need to make use of the functions -<b>pcre_jit_stack_alloc()</b>, <b>pcre_jit_stack_free()</b>, and -<b>pcre_assign_jit_stack()</b> in order to control the JIT code's memory usage. -</P> -<P> -From release 8.32 there is also a direct interface for JIT execution, which -gives improved performance. The JIT-specific functions are discussed in the -<a href="pcrejit.html"><b>pcrejit</b></a> -documentation. -</P> -<P> -A second matching function, <b>pcre_dfa_exec()</b>, which is not -Perl-compatible, is also provided. This uses a different algorithm for the -matching. The alternative algorithm finds all possible matches (at a given -point in the subject), and scans the subject just once (unless there are -lookbehind assertions). However, this algorithm does not return captured -substrings. A description of the two matching algorithms and their advantages -and disadvantages is given in the -<a href="pcrematching.html"><b>pcrematching</b></a> -documentation. -</P> -<P> -In addition to the main compiling and matching functions, there are convenience -functions for extracting captured substrings from a subject string that is -matched by <b>pcre_exec()</b>. They are: -<pre> - <b>pcre_copy_substring()</b> - <b>pcre_copy_named_substring()</b> - <b>pcre_get_substring()</b> - <b>pcre_get_named_substring()</b> - <b>pcre_get_substring_list()</b> - <b>pcre_get_stringnumber()</b> - <b>pcre_get_stringtable_entries()</b> -</pre> -<b>pcre_free_substring()</b> and <b>pcre_free_substring_list()</b> are also -provided, to free the memory used for extracted strings. -</P> -<P> -The function <b>pcre_maketables()</b> is used to build a set of character tables -in the current locale for passing to <b>pcre_compile()</b>, <b>pcre_exec()</b>, -or <b>pcre_dfa_exec()</b>. This is an optional facility that is provided for -specialist use. Most commonly, no special tables are passed, in which case -internal tables that are generated when PCRE is built are used. -</P> -<P> -The function <b>pcre_fullinfo()</b> is used to find out information about a -compiled pattern. The function <b>pcre_version()</b> returns a pointer to a -string containing the version of PCRE and its date of release. -</P> -<P> -The function <b>pcre_refcount()</b> maintains a reference count in a data block -containing a compiled pattern. This is provided for the benefit of -object-oriented applications. -</P> -<P> -The global variables <b>pcre_malloc</b> and <b>pcre_free</b> initially contain -the entry points of the standard <b>malloc()</b> and <b>free()</b> functions, -respectively. PCRE calls the memory management functions via these variables, -so a calling program can replace them if it wishes to intercept the calls. This -should be done before calling any PCRE functions. -</P> -<P> -The global variables <b>pcre_stack_malloc</b> and <b>pcre_stack_free</b> are also -indirections to memory management functions. These special functions are used -only when PCRE is compiled to use the heap for remembering data, instead of -recursive function calls, when running the <b>pcre_exec()</b> function. See the -<a href="pcrebuild.html"><b>pcrebuild</b></a> -documentation for details of how to do this. It is a non-standard way of -building PCRE, for use in environments that have limited stacks. Because of the -greater use of memory management, it runs more slowly. Separate functions are -provided so that special-purpose external code can be used for this case. When -used, these functions are always called in a stack-like manner (last obtained, -first freed), and always for memory blocks of the same size. There is a -discussion about PCRE's stack usage in the -<a href="pcrestack.html"><b>pcrestack</b></a> -documentation. -</P> -<P> -The global variable <b>pcre_callout</b> initially contains NULL. It can be set -by the caller to a "callout" function, which PCRE will then call at specified -points during a matching operation. Details are given in the -<a href="pcrecallout.html"><b>pcrecallout</b></a> -documentation. -</P> -<P> -The global variable <b>pcre_stack_guard</b> initially contains NULL. It can be -set by the caller to a function that is called by PCRE whenever it starts -to compile a parenthesized part of a pattern. When parentheses are nested, PCRE -uses recursive function calls, which use up the system stack. This function is -provided so that applications with restricted stacks can force a compilation -error if the stack runs out. The function should return zero if all is well, or -non-zero to force an error. -<a name="newlines"></a></P> -<br><a name="SEC7" href="#TOC1">NEWLINES</a><br> -<P> -PCRE supports five different conventions for indicating line breaks in -strings: a single CR (carriage return) character, a single LF (linefeed) -character, the two-character sequence CRLF, any of the three preceding, or any -Unicode newline sequence. The Unicode newline sequences are the three just -mentioned, plus the single characters VT (vertical tab, U+000B), FF (form feed, -U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS -(paragraph separator, U+2029). -</P> -<P> -Each of the first three conventions is used by at least one operating system as -its standard newline sequence. When PCRE is built, a default can be specified. -The default default is LF, which is the Unix standard. When PCRE is run, the -default can be overridden, either when a pattern is compiled, or when it is -matched. -</P> -<P> -At compile time, the newline convention can be specified by the <i>options</i> -argument of <b>pcre_compile()</b>, or it can be specified by special text at the -start of the pattern itself; this overrides any other settings. See the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -page for details of the special character sequences. -</P> -<P> -In the PCRE documentation the word "newline" is used to mean "the character or -pair of characters that indicate a line break". The choice of newline -convention affects the handling of the dot, circumflex, and dollar -metacharacters, the handling of #-comments in /x mode, and, when CRLF is a -recognized line ending sequence, the match position advancement for a -non-anchored pattern. There is more detail about this in the -<a href="#execoptions">section on <b>pcre_exec()</b> options</a> -below. -</P> -<P> -The choice of newline convention does not affect the interpretation of -the \n or \r escape sequences, nor does it affect what \R matches, which is -controlled in a similar way, but by separate options. -</P> -<br><a name="SEC8" href="#TOC1">MULTITHREADING</a><br> -<P> -The PCRE functions can be used in multi-threading applications, with the -proviso that the memory management functions pointed to by <b>pcre_malloc</b>, -<b>pcre_free</b>, <b>pcre_stack_malloc</b>, and <b>pcre_stack_free</b>, and the -callout and stack-checking functions pointed to by <b>pcre_callout</b> and -<b>pcre_stack_guard</b>, are shared by all threads. -</P> -<P> -The compiled form of a regular expression is not altered during matching, so -the same compiled pattern can safely be used by several threads at once. -</P> -<P> -If the just-in-time optimization feature is being used, it needs separate -memory stack areas for each thread. See the -<a href="pcrejit.html"><b>pcrejit</b></a> -documentation for more details. -</P> -<br><a name="SEC9" href="#TOC1">SAVING PRECOMPILED PATTERNS FOR LATER USE</a><br> -<P> -The compiled form of a regular expression can be saved and re-used at a later -time, possibly by a different program, and even on a host other than the one on -which it was compiled. Details are given in the -<a href="pcreprecompile.html"><b>pcreprecompile</b></a> -documentation, which includes a description of the -<b>pcre_pattern_to_host_byte_order()</b> function. However, compiling a regular -expression with one version of PCRE for use with a different version is not -guaranteed to work and may cause crashes. -</P> -<br><a name="SEC10" href="#TOC1">CHECKING BUILD-TIME OPTIONS</a><br> -<P> -<b>int pcre_config(int <i>what</i>, void *<i>where</i>);</b> -</P> -<P> -The function <b>pcre_config()</b> makes it possible for a PCRE client to -discover which optional features have been compiled into the PCRE library. The -<a href="pcrebuild.html"><b>pcrebuild</b></a> -documentation has more details about these optional features. -</P> -<P> -The first argument for <b>pcre_config()</b> is an integer, specifying which -information is required; the second argument is a pointer to a variable into -which the information is placed. The returned value is zero on success, or the -negative error code PCRE_ERROR_BADOPTION if the value in the first argument is -not recognized. The following information is available: -<pre> - PCRE_CONFIG_UTF8 -</pre> -The output is an integer that is set to one if UTF-8 support is available; -otherwise it is set to zero. This value should normally be given to the 8-bit -version of this function, <b>pcre_config()</b>. If it is given to the 16-bit -or 32-bit version of this function, the result is PCRE_ERROR_BADOPTION. -<pre> - PCRE_CONFIG_UTF16 -</pre> -The output is an integer that is set to one if UTF-16 support is available; -otherwise it is set to zero. This value should normally be given to the 16-bit -version of this function, <b>pcre16_config()</b>. If it is given to the 8-bit -or 32-bit version of this function, the result is PCRE_ERROR_BADOPTION. -<pre> - PCRE_CONFIG_UTF32 -</pre> -The output is an integer that is set to one if UTF-32 support is available; -otherwise it is set to zero. This value should normally be given to the 32-bit -version of this function, <b>pcre32_config()</b>. If it is given to the 8-bit -or 16-bit version of this function, the result is PCRE_ERROR_BADOPTION. -<pre> - PCRE_CONFIG_UNICODE_PROPERTIES -</pre> -The output is an integer that is set to one if support for Unicode character -properties is available; otherwise it is set to zero. -<pre> - PCRE_CONFIG_JIT -</pre> -The output is an integer that is set to one if support for just-in-time -compiling is available; otherwise it is set to zero. -<pre> - PCRE_CONFIG_JITTARGET -</pre> -The output is a pointer to a zero-terminated "const char *" string. If JIT -support is available, the string contains the name of the architecture for -which the JIT compiler is configured, for example "x86 32bit (little endian + -unaligned)". If JIT support is not available, the result is NULL. -<pre> - PCRE_CONFIG_NEWLINE -</pre> -The output is an integer whose value specifies the default character sequence -that is recognized as meaning "newline". The values that are supported in -ASCII/Unicode environments are: 10 for LF, 13 for CR, 3338 for CRLF, -2 for -ANYCRLF, and -1 for ANY. In EBCDIC environments, CR, ANYCRLF, and ANY yield the -same values. However, the value for LF is normally 21, though some EBCDIC -environments use 37. The corresponding values for CRLF are 3349 and 3365. The -default should normally correspond to the standard sequence for your operating -system. -<pre> - PCRE_CONFIG_BSR -</pre> -The output is an integer whose value indicates what character sequences the \R -escape sequence matches by default. A value of 0 means that \R matches any -Unicode line ending sequence; a value of 1 means that \R matches only CR, LF, -or CRLF. The default can be overridden when a pattern is compiled or matched. -<pre> - PCRE_CONFIG_LINK_SIZE -</pre> -The output is an integer that contains the number of bytes used for internal -linkage in compiled regular expressions. For the 8-bit library, the value can -be 2, 3, or 4. For the 16-bit library, the value is either 2 or 4 and is still -a number of bytes. For the 32-bit library, the value is either 2 or 4 and is -still a number of bytes. The default value of 2 is sufficient for all but the -most massive patterns, since it allows the compiled pattern to be up to 64K in -size. Larger values allow larger regular expressions to be compiled, at the -expense of slower matching. -<pre> - PCRE_CONFIG_POSIX_MALLOC_THRESHOLD -</pre> -The output is an integer that contains the threshold above which the POSIX -interface uses <b>malloc()</b> for output vectors. Further details are given in -the -<a href="pcreposix.html"><b>pcreposix</b></a> -documentation. -<pre> - PCRE_CONFIG_PARENS_LIMIT -</pre> -The output is a long integer that gives the maximum depth of nesting of -parentheses (of any kind) in a pattern. This limit is imposed to cap the amount -of system stack used when a pattern is compiled. It is specified when PCRE is -built; the default is 250. This limit does not take into account the stack that -may already be used by the calling application. For finer control over -compilation stack usage, you can set a pointer to an external checking function -in <b>pcre_stack_guard</b>. -<pre> - PCRE_CONFIG_MATCH_LIMIT -</pre> -The output is a long integer that gives the default limit for the number of -internal matching function calls in a <b>pcre_exec()</b> execution. Further -details are given with <b>pcre_exec()</b> below. -<pre> - PCRE_CONFIG_MATCH_LIMIT_RECURSION -</pre> -The output is a long integer that gives the default limit for the depth of -recursion when calling the internal matching function in a <b>pcre_exec()</b> -execution. Further details are given with <b>pcre_exec()</b> below. -<pre> - PCRE_CONFIG_STACKRECURSE -</pre> -The output is an integer that is set to one if internal recursion when running -<b>pcre_exec()</b> is implemented by recursive function calls that use the stack -to remember their state. This is the usual way that PCRE is compiled. The -output is zero if PCRE was compiled to use blocks of data on the heap instead -of recursive function calls. In this case, <b>pcre_stack_malloc</b> and -<b>pcre_stack_free</b> are called to manage memory blocks on the heap, thus -avoiding the use of the stack. -</P> -<br><a name="SEC11" href="#TOC1">COMPILING A PATTERN</a><br> -<P> -<b>pcre *pcre_compile(const char *<i>pattern</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -<br> -<br> -<b>pcre *pcre_compile2(const char *<i>pattern</i>, int <i>options</i>,</b> -<b> int *<i>errorcodeptr</i>,</b> -<b> const char **<i>errptr</i>, int *<i>erroffset</i>,</b> -<b> const unsigned char *<i>tableptr</i>);</b> -</P> -<P> -Either of the functions <b>pcre_compile()</b> or <b>pcre_compile2()</b> can be -called to compile a pattern into an internal form. The only difference between -the two interfaces is that <b>pcre_compile2()</b> has an additional argument, -<i>errorcodeptr</i>, via which a numerical error code can be returned. To avoid -too much repetition, we refer just to <b>pcre_compile()</b> below, but the -information applies equally to <b>pcre_compile2()</b>. -</P> -<P> -The pattern is a C string terminated by a binary zero, and is passed in the -<i>pattern</i> argument. A pointer to a single block of memory that is obtained -via <b>pcre_malloc</b> is returned. This contains the compiled code and related -data. The <b>pcre</b> type is defined for the returned block; this is a typedef -for a structure whose contents are not externally defined. It is up to the -caller to free the memory (via <b>pcre_free</b>) when it is no longer required. -</P> -<P> -Although the compiled code of a PCRE regex is relocatable, that is, it does not -depend on memory location, the complete <b>pcre</b> data block is not -fully relocatable, because it may contain a copy of the <i>tableptr</i> -argument, which is an address (see below). -</P> -<P> -The <i>options</i> argument contains various bit settings that affect the -compilation. It should be zero if no options are required. The available -options are described below. Some of them (in particular, those that are -compatible with Perl, but some others as well) can also be set and unset from -within the pattern (see the detailed description in the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -documentation). For those options that can be different in different parts of -the pattern, the contents of the <i>options</i> argument specifies their -settings at the start of compilation and execution. The PCRE_ANCHORED, -PCRE_BSR_<i>xxx</i>, PCRE_NEWLINE_<i>xxx</i>, PCRE_NO_UTF8_CHECK, and -PCRE_NO_START_OPTIMIZE options can be set at the time of matching as well as at -compile time. -</P> -<P> -If <i>errptr</i> is NULL, <b>pcre_compile()</b> returns NULL immediately. -Otherwise, if compilation of a pattern fails, <b>pcre_compile()</b> returns -NULL, and sets the variable pointed to by <i>errptr</i> to point to a textual -error message. This is a static string that is part of the library. You must -not try to free it. Normally, the offset from the start of the pattern to the -data unit that was being processed when the error was discovered is placed in -the variable pointed to by <i>erroffset</i>, which must not be NULL (if it is, -an immediate error is given). However, for an invalid UTF-8 or UTF-16 string, -the offset is that of the first data unit of the failing character. -</P> -<P> -Some errors are not detected until the whole pattern has been scanned; in these -cases, the offset passed back is the length of the pattern. Note that the -offset is in data units, not characters, even in a UTF mode. It may sometimes -point into the middle of a UTF-8 or UTF-16 character. -</P> -<P> -If <b>pcre_compile2()</b> is used instead of <b>pcre_compile()</b>, and the -<i>errorcodeptr</i> argument is not NULL, a non-zero error code number is -returned via this argument in the event of an error. This is in addition to the -textual error message. Error codes and messages are listed below. -</P> -<P> -If the final argument, <i>tableptr</i>, is NULL, PCRE uses a default set of -character tables that are built when PCRE is compiled, using the default C -locale. Otherwise, <i>tableptr</i> must be an address that is the result of a -call to <b>pcre_maketables()</b>. This value is stored with the compiled -pattern, and used again by <b>pcre_exec()</b> and <b>pcre_dfa_exec()</b> when the -pattern is matched. For more discussion, see the section on locale support -below. -</P> -<P> -This code fragment shows a typical straightforward call to <b>pcre_compile()</b>: -<pre> - pcre *re; - const char *error; - int erroffset; - re = pcre_compile( - "^A.*Z", /* the pattern */ - 0, /* default options */ - &error, /* for error message */ - &erroffset, /* for error offset */ - NULL); /* use default character tables */ -</pre> -The following names for option bits are defined in the <b>pcre.h</b> header -file: -<pre> - PCRE_ANCHORED -</pre> -If this bit is set, the pattern is forced to be "anchored", that is, it is -constrained to match only at the first matching point in the string that is -being searched (the "subject string"). This effect can also be achieved by -appropriate constructs in the pattern itself, which is the only way to do it in -Perl. -<pre> - PCRE_AUTO_CALLOUT -</pre> -If this bit is set, <b>pcre_compile()</b> automatically inserts callout items, -all with number 255, before each pattern item. For discussion of the callout -facility, see the -<a href="pcrecallout.html"><b>pcrecallout</b></a> -documentation. -<pre> - PCRE_BSR_ANYCRLF - PCRE_BSR_UNICODE -</pre> -These options (which are mutually exclusive) control what the \R escape -sequence matches. The choice is either to match only CR, LF, or CRLF, or to -match any Unicode newline sequence. The default is specified when PCRE is -built. It can be overridden from within the pattern, or by setting an option -when a compiled pattern is matched. -<pre> - PCRE_CASELESS -</pre> -If this bit is set, letters in the pattern match both upper and lower case -letters. It is equivalent to Perl's /i option, and it can be changed within a -pattern by a (?i) option setting. In UTF-8 mode, PCRE always understands the -concept of case for characters whose values are less than 128, so caseless -matching is always possible. For characters with higher values, the concept of -case is supported if PCRE is compiled with Unicode property support, but not -otherwise. If you want to use caseless matching for characters 128 and above, -you must ensure that PCRE is compiled with Unicode property support as well as -with UTF-8 support. -<pre> - PCRE_DOLLAR_ENDONLY -</pre> -If this bit is set, a dollar metacharacter in the pattern matches only at the -end of the subject string. Without this option, a dollar also matches -immediately before a newline at the end of the string (but not before any other -newlines). The PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is set. -There is no equivalent to this option in Perl, and no way to set it within a -pattern. -<pre> - PCRE_DOTALL -</pre> -If this bit is set, a dot metacharacter in the pattern matches a character of -any value, including one that indicates a newline. However, it only ever -matches one character, even if newlines are coded as CRLF. Without this option, -a dot does not match when the current position is at a newline. This option is -equivalent to Perl's /s option, and it can be changed within a pattern by a -(?s) option setting. A negative class such as [^a] always matches newline -characters, independent of the setting of this option. -<pre> - PCRE_DUPNAMES -</pre> -If this bit is set, names used to identify capturing subpatterns need not be -unique. This can be helpful for certain types of pattern when it is known that -only one instance of the named subpattern can ever be matched. There are more -details of named subpatterns below; see also the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -documentation. -<pre> - PCRE_EXTENDED -</pre> -If this bit is set, most white space characters in the pattern are totally -ignored except when escaped or inside a character class. However, white space -is not allowed within sequences such as (?> that introduce various -parenthesized subpatterns, nor within a numerical quantifier such as {1,3}. -However, ignorable white space is permitted between an item and a following -quantifier and between a quantifier and a following + that indicates -possessiveness. -</P> -<P> -White space did not used to include the VT character (code 11), because Perl -did not treat this character as white space. However, Perl changed at release -5.18, so PCRE followed at release 8.34, and VT is now treated as white space. -</P> -<P> -PCRE_EXTENDED also causes characters between an unescaped # outside a character -class and the next newline, inclusive, to be ignored. PCRE_EXTENDED is -equivalent to Perl's /x option, and it can be changed within a pattern by a -(?x) option setting. -</P> -<P> -Which characters are interpreted as newlines is controlled by the options -passed to <b>pcre_compile()</b> or by a special sequence at the start of the -pattern, as described in the section entitled -<a href="pcrepattern.html#newlines">"Newline conventions"</a> -in the <b>pcrepattern</b> documentation. Note that the end of this type of -comment is a literal newline sequence in the pattern; escape sequences that -happen to represent a newline do not count. -</P> -<P> -This option makes it possible to include comments inside complicated patterns. -Note, however, that this applies only to data characters. White space characters -may never appear within special character sequences in a pattern, for example -within the sequence (?( that introduces a conditional subpattern. -<pre> - PCRE_EXTRA -</pre> -This option was invented in order to turn on additional functionality of PCRE -that is incompatible with Perl, but it is currently of very little use. When -set, any backslash in a pattern that is followed by a letter that has no -special meaning causes an error, thus reserving these combinations for future -expansion. By default, as in Perl, a backslash followed by a letter with no -special meaning is treated as a literal. (Perl can, however, be persuaded to -give an error for this, by running it with the -w option.) There are at present -no other features controlled by this option. It can also be set by a (?X) -option setting within a pattern. -<pre> - PCRE_FIRSTLINE -</pre> -If this option is set, an unanchored pattern is required to match before or at -the first newline in the subject string, though the matched text may continue -over the newline. -<pre> - PCRE_JAVASCRIPT_COMPAT -</pre> -If this option is set, PCRE's behaviour is changed in some ways so that it is -compatible with JavaScript rather than Perl. The changes are as follows: -</P> -<P> -(1) A lone closing square bracket in a pattern causes a compile-time error, -because this is illegal in JavaScript (by default it is treated as a data -character). Thus, the pattern AB]CD becomes illegal when this option is set. -</P> -<P> -(2) At run time, a back reference to an unset subpattern group matches an empty -string (by default this causes the current matching alternative to fail). A -pattern such as (\1)(a) succeeds when this option is set (assuming it can find -an "a" in the subject), whereas it fails by default, for Perl compatibility. -</P> -<P> -(3) \U matches an upper case "U" character; by default \U causes a compile -time error (Perl uses \U to upper case subsequent characters). -</P> -<P> -(4) \u matches a lower case "u" character unless it is followed by four -hexadecimal digits, in which case the hexadecimal number defines the code point -to match. By default, \u causes a compile time error (Perl uses it to upper -case the following character). -</P> -<P> -(5) \x matches a lower case "x" character unless it is followed by two -hexadecimal digits, in which case the hexadecimal number defines the code point -to match. By default, as in Perl, a hexadecimal number is always expected after -\x, but it may have zero, one, or two digits (so, for example, \xz matches a -binary zero character followed by z). -<pre> - PCRE_MULTILINE -</pre> -By default, for the purposes of matching "start of line" and "end of line", -PCRE treats the subject string as consisting of a single line of characters, -even if it actually contains newlines. The "start of line" metacharacter (^) -matches only at the start of the string, and the "end of line" metacharacter -($) matches only at the end of the string, or before a terminating newline -(except when PCRE_DOLLAR_ENDONLY is set). Note, however, that unless -PCRE_DOTALL is set, the "any character" metacharacter (.) does not match at a -newline. This behaviour (for ^, $, and dot) is the same as Perl. -</P> -<P> -When PCRE_MULTILINE it is set, the "start of line" and "end of line" constructs -match immediately following or immediately before internal newlines in the -subject string, respectively, as well as at the very start and end. This is -equivalent to Perl's /m option, and it can be changed within a pattern by a -(?m) option setting. If there are no newlines in a subject string, or no -occurrences of ^ or $ in a pattern, setting PCRE_MULTILINE has no effect. -<pre> - PCRE_NEVER_UTF -</pre> -This option locks out interpretation of the pattern as UTF-8 (or UTF-16 or -UTF-32 in the 16-bit and 32-bit libraries). In particular, it prevents the -creator of the pattern from switching to UTF interpretation by starting the -pattern with (*UTF). This may be useful in applications that process patterns -from external sources. The combination of PCRE_UTF8 and PCRE_NEVER_UTF also -causes an error. -<pre> - PCRE_NEWLINE_CR - PCRE_NEWLINE_LF - PCRE_NEWLINE_CRLF - PCRE_NEWLINE_ANYCRLF - PCRE_NEWLINE_ANY -</pre> -These options override the default newline definition that was chosen when PCRE -was built. Setting the first or the second specifies that a newline is -indicated by a single character (CR or LF, respectively). Setting -PCRE_NEWLINE_CRLF specifies that a newline is indicated by the two-character -CRLF sequence. Setting PCRE_NEWLINE_ANYCRLF specifies that any of the three -preceding sequences should be recognized. Setting PCRE_NEWLINE_ANY specifies -that any Unicode newline sequence should be recognized. -</P> -<P> -In an ASCII/Unicode environment, the Unicode newline sequences are the three -just mentioned, plus the single characters VT (vertical tab, U+000B), FF (form -feed, U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS -(paragraph separator, U+2029). For the 8-bit library, the last two are -recognized only in UTF-8 mode. -</P> -<P> -When PCRE is compiled to run in an EBCDIC (mainframe) environment, the code for -CR is 0x0d, the same as ASCII. However, the character code for LF is normally -0x15, though in some EBCDIC environments 0x25 is used. Whichever of these is -not LF is made to correspond to Unicode's NEL character. EBCDIC codes are all -less than 256. For more details, see the -<a href="pcrebuild.html"><b>pcrebuild</b></a> -documentation. -</P> -<P> -The newline setting in the options word uses three bits that are treated -as a number, giving eight possibilities. Currently only six are used (default -plus the five values above). This means that if you set more than one newline -option, the combination may or may not be sensible. For example, -PCRE_NEWLINE_CR with PCRE_NEWLINE_LF is equivalent to PCRE_NEWLINE_CRLF, but -other combinations may yield unused numbers and cause an error. -</P> -<P> -The only time that a line break in a pattern is specially recognized when -compiling is when PCRE_EXTENDED is set. CR and LF are white space characters, -and so are ignored in this mode. Also, an unescaped # outside a character class -indicates a comment that lasts until after the next line break sequence. In -other circumstances, line break sequences in patterns are treated as literal -data. -</P> -<P> -The newline option that is set at compile time becomes the default that is used -for <b>pcre_exec()</b> and <b>pcre_dfa_exec()</b>, but it can be overridden. -<pre> - PCRE_NO_AUTO_CAPTURE -</pre> -If this option is set, it disables the use of numbered capturing parentheses in -the pattern. Any opening parenthesis that is not followed by ? behaves as if it -were followed by ?: but named parentheses can still be used for capturing (and -they acquire numbers in the usual way). There is no equivalent of this option -in Perl. -<pre> - PCRE_NO_AUTO_POSSESS -</pre> -If this option is set, it disables "auto-possessification". This is an -optimization that, for example, turns a+b into a++b in order to avoid -backtracks into a+ that can never be successful. However, if callouts are in -use, auto-possessification means that some of them are never taken. You can set -this option if you want the matching functions to do a full unoptimized search -and run all the callouts, but it is mainly provided for testing purposes. -<pre> - PCRE_NO_START_OPTIMIZE -</pre> -This is an option that acts at matching time; that is, it is really an option -for <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b>. If it is set at compile time, -it is remembered with the compiled pattern and assumed at matching time. This -is necessary if you want to use JIT execution, because the JIT compiler needs -to know whether or not this option is set. For details see the discussion of -PCRE_NO_START_OPTIMIZE -<a href="#execoptions">below.</a> -<pre> - PCRE_UCP -</pre> -This option changes the way PCRE processes \B, \b, \D, \d, \S, \s, \W, -\w, and some of the POSIX character classes. By default, only ASCII characters -are recognized, but if PCRE_UCP is set, Unicode properties are used instead to -classify characters. More details are given in the section on -<a href="pcre.html#genericchartypes">generic character types</a> -in the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -page. If you set PCRE_UCP, matching one of the items it affects takes much -longer. The option is available only if PCRE has been compiled with Unicode -property support. -<pre> - PCRE_UNGREEDY -</pre> -This option inverts the "greediness" of the quantifiers so that they are not -greedy by default, but become greedy if followed by "?". It is not compatible -with Perl. It can also be set by a (?U) option setting within the pattern. -<pre> - PCRE_UTF8 -</pre> -This option causes PCRE to regard both the pattern and the subject as strings -of UTF-8 characters instead of single-byte strings. However, it is available -only when PCRE is built to include UTF support. If not, the use of this option -provokes an error. Details of how this option changes the behaviour of PCRE are -given in the -<a href="pcreunicode.html"><b>pcreunicode</b></a> -page. -<pre> - PCRE_NO_UTF8_CHECK -</pre> -When PCRE_UTF8 is set, the validity of the pattern as a UTF-8 string is -automatically checked. There is a discussion about the -<a href="pcreunicode.html#utf8strings">validity of UTF-8 strings</a> -in the -<a href="pcreunicode.html"><b>pcreunicode</b></a> -page. If an invalid UTF-8 sequence is found, <b>pcre_compile()</b> returns an -error. If you already know that your pattern is valid, and you want to skip -this check for performance reasons, you can set the PCRE_NO_UTF8_CHECK option. -When it is set, the effect of passing an invalid UTF-8 string as a pattern is -undefined. It may cause your program to crash or loop. Note that this option -can also be passed to <b>pcre_exec()</b> and <b>pcre_dfa_exec()</b>, to suppress -the validity checking of subject strings only. If the same string is being -matched many times, the option can be safely set for the second and subsequent -matchings to improve performance. -</P> -<br><a name="SEC12" href="#TOC1">COMPILATION ERROR CODES</a><br> -<P> -The following table lists the error codes than may be returned by -<b>pcre_compile2()</b>, along with the error messages that may be returned by -both compiling functions. Note that error messages are always 8-bit ASCII -strings, even in 16-bit or 32-bit mode. As PCRE has developed, some error codes -have fallen out of use. To avoid confusion, they have not been re-used. -<pre> - 0 no error - 1 \ at end of pattern - 2 \c at end of pattern - 3 unrecognized character follows \ - 4 numbers out of order in {} quantifier - 5 number too big in {} quantifier - 6 missing terminating ] for character class - 7 invalid escape sequence in character class - 8 range out of order in character class - 9 nothing to repeat - 10 [this code is not in use] - 11 internal error: unexpected repeat - 12 unrecognized character after (? or (?- - 13 POSIX named classes are supported only within a class - 14 missing ) - 15 reference to non-existent subpattern - 16 erroffset passed as NULL - 17 unknown option bit(s) set - 18 missing ) after comment - 19 [this code is not in use] - 20 regular expression is too large - 21 failed to get memory - 22 unmatched parentheses - 23 internal error: code overflow - 24 unrecognized character after (?< - 25 lookbehind assertion is not fixed length - 26 malformed number or name after (?( - 27 conditional group contains more than two branches - 28 assertion expected after (?( - 29 (?R or (?[+-]digits must be followed by ) - 30 unknown POSIX class name - 31 POSIX collating elements are not supported - 32 this version of PCRE is compiled without UTF support - 33 [this code is not in use] - 34 character value in \x{} or \o{} is too large - 35 invalid condition (?(0) - 36 \C not allowed in lookbehind assertion - 37 PCRE does not support \L, \l, \N{name}, \U, or \u - 38 number after (?C is > 255 - 39 closing ) for (?C expected - 40 recursive call could loop indefinitely - 41 unrecognized character after (?P - 42 syntax error in subpattern name (missing terminator) - 43 two named subpatterns have the same name - 44 invalid UTF-8 string (specifically UTF-8) - 45 support for \P, \p, and \X has not been compiled - 46 malformed \P or \p sequence - 47 unknown property name after \P or \p - 48 subpattern name is too long (maximum 32 characters) - 49 too many named subpatterns (maximum 10000) - 50 [this code is not in use] - 51 octal value is greater than \377 in 8-bit non-UTF-8 mode - 52 internal error: overran compiling workspace - 53 internal error: previously-checked referenced subpattern - not found - 54 DEFINE group contains more than one branch - 55 repeating a DEFINE group is not allowed - 56 inconsistent NEWLINE options - 57 \g is not followed by a braced, angle-bracketed, or quoted - name/number or by a plain number - 58 a numbered reference must not be zero - 59 an argument is not allowed for (*ACCEPT), (*FAIL), or (*COMMIT) - 60 (*VERB) not recognized or malformed - 61 number is too big - 62 subpattern name expected - 63 digit expected after (?+ - 64 ] is an invalid data character in JavaScript compatibility mode - 65 different names for subpatterns of the same number are - not allowed - 66 (*MARK) must have an argument - 67 this version of PCRE is not compiled with Unicode property - support - 68 \c must be followed by an ASCII character - 69 \k is not followed by a braced, angle-bracketed, or quoted name - 70 internal error: unknown opcode in find_fixedlength() - 71 \N is not supported in a class - 72 too many forward references - 73 disallowed Unicode code point (>= 0xd800 && <= 0xdfff) - 74 invalid UTF-16 string (specifically UTF-16) - 75 name is too long in (*MARK), (*PRUNE), (*SKIP), or (*THEN) - 76 character value in \u.... sequence is too large - 77 invalid UTF-32 string (specifically UTF-32) - 78 setting UTF is disabled by the application - 79 non-hex character in \x{} (closing brace missing?) - 80 non-octal character in \o{} (closing brace missing?) - 81 missing opening brace after \o - 82 parentheses are too deeply nested - 83 invalid range in character class - 84 group name must start with a non-digit - 85 parentheses are too deeply nested (stack check) -</pre> -The numbers 32 and 10000 in errors 48 and 49 are defaults; different values may -be used if the limits were changed when PCRE was built. -<a name="studyingapattern"></a></P> -<br><a name="SEC13" href="#TOC1">STUDYING A PATTERN</a><br> -<P> -<b>pcre_extra *pcre_study(const pcre *<i>code</i>, int <i>options</i>,</b> -<b> const char **<i>errptr</i>);</b> -</P> -<P> -If a compiled pattern is going to be used several times, it is worth spending -more time analyzing it in order to speed up the time taken for matching. The -function <b>pcre_study()</b> takes a pointer to a compiled pattern as its first -argument. If studying the pattern produces additional information that will -help speed up matching, <b>pcre_study()</b> returns a pointer to a -<b>pcre_extra</b> block, in which the <i>study_data</i> field points to the -results of the study. -</P> -<P> -The returned value from <b>pcre_study()</b> can be passed directly to -<b>pcre_exec()</b> or <b>pcre_dfa_exec()</b>. However, a <b>pcre_extra</b> block -also contains other fields that can be set by the caller before the block is -passed; these are described -<a href="#extradata">below</a> -in the section on matching a pattern. -</P> -<P> -If studying the pattern does not produce any useful information, -<b>pcre_study()</b> returns NULL by default. In that circumstance, if the -calling program wants to pass any of the other fields to <b>pcre_exec()</b> or -<b>pcre_dfa_exec()</b>, it must set up its own <b>pcre_extra</b> block. However, -if <b>pcre_study()</b> is called with the PCRE_STUDY_EXTRA_NEEDED option, it -returns a <b>pcre_extra</b> block even if studying did not find any additional -information. It may still return NULL, however, if an error occurs in -<b>pcre_study()</b>. -</P> -<P> -The second argument of <b>pcre_study()</b> contains option bits. There are three -further options in addition to PCRE_STUDY_EXTRA_NEEDED: -<pre> - PCRE_STUDY_JIT_COMPILE - PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE - PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE -</pre> -If any of these are set, and the just-in-time compiler is available, the -pattern is further compiled into machine code that executes much faster than -the <b>pcre_exec()</b> interpretive matching function. If the just-in-time -compiler is not available, these options are ignored. All undefined bits in the -<i>options</i> argument must be zero. -</P> -<P> -JIT compilation is a heavyweight optimization. It can take some time for -patterns to be analyzed, and for one-off matches and simple patterns the -benefit of faster execution might be offset by a much slower study time. -Not all patterns can be optimized by the JIT compiler. For those that cannot be -handled, matching automatically falls back to the <b>pcre_exec()</b> -interpreter. For more details, see the -<a href="pcrejit.html"><b>pcrejit</b></a> -documentation. -</P> -<P> -The third argument for <b>pcre_study()</b> is a pointer for an error message. If -studying succeeds (even if no data is returned), the variable it points to is -set to NULL. Otherwise it is set to point to a textual error message. This is a -static string that is part of the library. You must not try to free it. You -should test the error pointer for NULL after calling <b>pcre_study()</b>, to be -sure that it has run successfully. -</P> -<P> -When you are finished with a pattern, you can free the memory used for the -study data by calling <b>pcre_free_study()</b>. This function was added to the -API for release 8.20. For earlier versions, the memory could be freed with -<b>pcre_free()</b>, just like the pattern itself. This will still work in cases -where JIT optimization is not used, but it is advisable to change to the new -function when convenient. -</P> -<P> -This is a typical way in which <b>pcre_study</b>() is used (except that in a -real application there should be tests for errors): -<pre> - int rc; - pcre *re; - pcre_extra *sd; - re = pcre_compile("pattern", 0, &error, &erroroffset, NULL); - sd = pcre_study( - re, /* result of pcre_compile() */ - 0, /* no options */ - &error); /* set to NULL or points to a message */ - rc = pcre_exec( /* see below for details of pcre_exec() options */ - re, sd, "subject", 7, 0, 0, ovector, 30); - ... - pcre_free_study(sd); - pcre_free(re); -</pre> -Studying a pattern does two things: first, a lower bound for the length of -subject string that is needed to match the pattern is computed. This does not -mean that there are any strings of that length that match, but it does -guarantee that no shorter strings match. The value is used to avoid wasting -time by trying to match strings that are shorter than the lower bound. You can -find out the value in a calling program via the <b>pcre_fullinfo()</b> function. -</P> -<P> -Studying a pattern is also useful for non-anchored patterns that do not have a -single fixed starting character. A bitmap of possible starting bytes is -created. This speeds up finding a position in the subject at which to start -matching. (In 16-bit mode, the bitmap is used for 16-bit values less than 256. -In 32-bit mode, the bitmap is used for 32-bit values less than 256.) -</P> -<P> -These two optimizations apply to both <b>pcre_exec()</b> and -<b>pcre_dfa_exec()</b>, and the information is also used by the JIT compiler. -The optimizations can be disabled by setting the PCRE_NO_START_OPTIMIZE option. -You might want to do this if your pattern contains callouts or (*MARK) and you -want to make use of these facilities in cases where matching fails. -</P> -<P> -PCRE_NO_START_OPTIMIZE can be specified at either compile time or execution -time. However, if PCRE_NO_START_OPTIMIZE is passed to <b>pcre_exec()</b>, (that -is, after any JIT compilation has happened) JIT execution is disabled. For JIT -execution to work with PCRE_NO_START_OPTIMIZE, the option must be set at -compile time. -</P> -<P> -There is a longer discussion of PCRE_NO_START_OPTIMIZE -<a href="#execoptions">below.</a> -<a name="localesupport"></a></P> -<br><a name="SEC14" href="#TOC1">LOCALE SUPPORT</a><br> -<P> -PCRE handles caseless matching, and determines whether characters are letters, -digits, or whatever, by reference to a set of tables, indexed by character -code point. When running in UTF-8 mode, or in the 16- or 32-bit libraries, this -applies only to characters with code points less than 256. By default, -higher-valued code points never match escapes such as \w or \d. However, if -PCRE is built with Unicode property support, all characters can be tested with -\p and \P, or, alternatively, the PCRE_UCP option can be set when a pattern -is compiled; this causes \w and friends to use Unicode property support -instead of the built-in tables. -</P> -<P> -The use of locales with Unicode is discouraged. If you are handling characters -with code points greater than 128, you should either use Unicode support, or -use locales, but not try to mix the two. -</P> -<P> -PCRE contains an internal set of tables that are used when the final argument -of <b>pcre_compile()</b> is NULL. These are sufficient for many applications. -Normally, the internal tables recognize only ASCII characters. However, when -PCRE is built, it is possible to cause the internal tables to be rebuilt in the -default "C" locale of the local system, which may cause them to be different. -</P> -<P> -The internal tables can always be overridden by tables supplied by the -application that calls PCRE. These may be created in a different locale from -the default. As more and more applications change to using Unicode, the need -for this locale support is expected to die away. -</P> -<P> -External tables are built by calling the <b>pcre_maketables()</b> function, -which has no arguments, in the relevant locale. The result can then be passed -to <b>pcre_compile()</b> as often as necessary. For example, to build and use -tables that are appropriate for the French locale (where accented characters -with values greater than 128 are treated as letters), the following code could -be used: -<pre> - setlocale(LC_CTYPE, "fr_FR"); - tables = pcre_maketables(); - re = pcre_compile(..., tables); -</pre> -The locale name "fr_FR" is used on Linux and other Unix-like systems; if you -are using Windows, the name for the French locale is "french". -</P> -<P> -When <b>pcre_maketables()</b> runs, the tables are built in memory that is -obtained via <b>pcre_malloc</b>. It is the caller's responsibility to ensure -that the memory containing the tables remains available for as long as it is -needed. -</P> -<P> -The pointer that is passed to <b>pcre_compile()</b> is saved with the compiled -pattern, and the same tables are used via this pointer by <b>pcre_study()</b> -and also by <b>pcre_exec()</b> and <b>pcre_dfa_exec()</b>. Thus, for any single -pattern, compilation, studying and matching all happen in the same locale, but -different patterns can be processed in different locales. -</P> -<P> -It is possible to pass a table pointer or NULL (indicating the use of the -internal tables) to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> (see the -discussion below in the section on matching a pattern). This facility is -provided for use with pre-compiled patterns that have been saved and reloaded. -Character tables are not saved with patterns, so if a non-standard table was -used at compile time, it must be provided again when the reloaded pattern is -matched. Attempting to use this facility to match a pattern in a different -locale from the one in which it was compiled is likely to lead to anomalous -(usually incorrect) results. -<a name="infoaboutpattern"></a></P> -<br><a name="SEC15" href="#TOC1">INFORMATION ABOUT A PATTERN</a><br> -<P> -<b>int pcre_fullinfo(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> -<b> int <i>what</i>, void *<i>where</i>);</b> -</P> -<P> -The <b>pcre_fullinfo()</b> function returns information about a compiled -pattern. It replaces the <b>pcre_info()</b> function, which was removed from the -library at version 8.30, after more than 10 years of obsolescence. -</P> -<P> -The first argument for <b>pcre_fullinfo()</b> is a pointer to the compiled -pattern. The second argument is the result of <b>pcre_study()</b>, or NULL if -the pattern was not studied. The third argument specifies which piece of -information is required, and the fourth argument is a pointer to a variable -to receive the data. The yield of the function is zero for success, or one of -the following negative numbers: -<pre> - PCRE_ERROR_NULL the argument <i>code</i> was NULL - the argument <i>where</i> was NULL - PCRE_ERROR_BADMAGIC the "magic number" was not found - PCRE_ERROR_BADENDIANNESS the pattern was compiled with different - endianness - PCRE_ERROR_BADOPTION the value of <i>what</i> was invalid - PCRE_ERROR_UNSET the requested field is not set -</pre> -The "magic number" is placed at the start of each compiled pattern as an simple -check against passing an arbitrary memory pointer. The endianness error can -occur if a compiled pattern is saved and reloaded on a different host. Here is -a typical call of <b>pcre_fullinfo()</b>, to obtain the length of the compiled -pattern: -<pre> - int rc; - size_t length; - rc = pcre_fullinfo( - re, /* result of pcre_compile() */ - sd, /* result of pcre_study(), or NULL */ - PCRE_INFO_SIZE, /* what is required */ - &length); /* where to put the data */ -</pre> -The possible values for the third argument are defined in <b>pcre.h</b>, and are -as follows: -<pre> - PCRE_INFO_BACKREFMAX -</pre> -Return the number of the highest back reference in the pattern. The fourth -argument should point to an <b>int</b> variable. Zero is returned if there are -no back references. -<pre> - PCRE_INFO_CAPTURECOUNT -</pre> -Return the number of capturing subpatterns in the pattern. The fourth argument -should point to an <b>int</b> variable. -<pre> - PCRE_INFO_DEFAULT_TABLES -</pre> -Return a pointer to the internal default character tables within PCRE. The -fourth argument should point to an <b>unsigned char *</b> variable. This -information call is provided for internal use by the <b>pcre_study()</b> -function. External callers can cause PCRE to use its internal tables by passing -a NULL table pointer. -<pre> - PCRE_INFO_FIRSTBYTE (deprecated) -</pre> -Return information about the first data unit of any matched string, for a -non-anchored pattern. The name of this option refers to the 8-bit library, -where data units are bytes. The fourth argument should point to an <b>int</b> -variable. Negative values are used for special cases. However, this means that -when the 32-bit library is in non-UTF-32 mode, the full 32-bit range of -characters cannot be returned. For this reason, this value is deprecated; use -PCRE_INFO_FIRSTCHARACTERFLAGS and PCRE_INFO_FIRSTCHARACTER instead. -</P> -<P> -If there is a fixed first value, for example, the letter "c" from a pattern -such as (cat|cow|coyote), its value is returned. In the 8-bit library, the -value is always less than 256. In the 16-bit library the value can be up to -0xffff. In the 32-bit library the value can be up to 0x10ffff. -</P> -<P> -If there is no fixed first value, and if either -<br> -<br> -(a) the pattern was compiled with the PCRE_MULTILINE option, and every branch -starts with "^", or -<br> -<br> -(b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not set -(if it were set, the pattern would be anchored), -<br> -<br> --1 is returned, indicating that the pattern matches only at the start of a -subject string or after any newline within the string. Otherwise -2 is -returned. For anchored patterns, -2 is returned. -<pre> - PCRE_INFO_FIRSTCHARACTER -</pre> -Return the value of the first data unit (non-UTF character) of any matched -string in the situation where PCRE_INFO_FIRSTCHARACTERFLAGS returns 1; -otherwise return 0. The fourth argument should point to an <b>uint_t</b> -variable. -</P> -<P> -In the 8-bit library, the value is always less than 256. In the 16-bit library -the value can be up to 0xffff. In the 32-bit library in UTF-32 mode the value -can be up to 0x10ffff, and up to 0xffffffff when not using UTF-32 mode. -<pre> - PCRE_INFO_FIRSTCHARACTERFLAGS -</pre> -Return information about the first data unit of any matched string, for a -non-anchored pattern. The fourth argument should point to an <b>int</b> -variable. -</P> -<P> -If there is a fixed first value, for example, the letter "c" from a pattern -such as (cat|cow|coyote), 1 is returned, and the character value can be -retrieved using PCRE_INFO_FIRSTCHARACTER. If there is no fixed first value, and -if either -<br> -<br> -(a) the pattern was compiled with the PCRE_MULTILINE option, and every branch -starts with "^", or -<br> -<br> -(b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not set -(if it were set, the pattern would be anchored), -<br> -<br> -2 is returned, indicating that the pattern matches only at the start of a -subject string or after any newline within the string. Otherwise 0 is -returned. For anchored patterns, 0 is returned. -<pre> - PCRE_INFO_FIRSTTABLE -</pre> -If the pattern was studied, and this resulted in the construction of a 256-bit -table indicating a fixed set of values for the first data unit in any matching -string, a pointer to the table is returned. Otherwise NULL is returned. The -fourth argument should point to an <b>unsigned char *</b> variable. -<pre> - PCRE_INFO_HASCRORLF -</pre> -Return 1 if the pattern contains any explicit matches for CR or LF characters, -otherwise 0. The fourth argument should point to an <b>int</b> variable. An -explicit match is either a literal CR or LF character, or \r or \n. -<pre> - PCRE_INFO_JCHANGED -</pre> -Return 1 if the (?J) or (?-J) option setting is used in the pattern, otherwise -0. The fourth argument should point to an <b>int</b> variable. (?J) and -(?-J) set and unset the local PCRE_DUPNAMES option, respectively. -<pre> - PCRE_INFO_JIT -</pre> -Return 1 if the pattern was studied with one of the JIT options, and -just-in-time compiling was successful. The fourth argument should point to an -<b>int</b> variable. A return value of 0 means that JIT support is not available -in this version of PCRE, or that the pattern was not studied with a JIT option, -or that the JIT compiler could not handle this particular pattern. See the -<a href="pcrejit.html"><b>pcrejit</b></a> -documentation for details of what can and cannot be handled. -<pre> - PCRE_INFO_JITSIZE -</pre> -If the pattern was successfully studied with a JIT option, return the size of -the JIT compiled code, otherwise return zero. The fourth argument should point -to a <b>size_t</b> variable. -<pre> - PCRE_INFO_LASTLITERAL -</pre> -Return the value of the rightmost literal data unit that must exist in any -matched string, other than at its start, if such a value has been recorded. The -fourth argument should point to an <b>int</b> variable. If there is no such -value, -1 is returned. For anchored patterns, a last literal value is recorded -only if it follows something of variable length. For example, for the pattern -/^a\d+z\d+/ the returned value is "z", but for /^a\dz\d/ the returned value -is -1. -</P> -<P> -Since for the 32-bit library using the non-UTF-32 mode, this function is unable -to return the full 32-bit range of characters, this value is deprecated; -instead the PCRE_INFO_REQUIREDCHARFLAGS and PCRE_INFO_REQUIREDCHAR values should -be used. -<pre> - PCRE_INFO_MATCH_EMPTY -</pre> -Return 1 if the pattern can match an empty string, otherwise 0. The fourth -argument should point to an <b>int</b> variable. -<pre> - PCRE_INFO_MATCHLIMIT -</pre> -If the pattern set a match limit by including an item of the form -(*LIMIT_MATCH=nnnn) at the start, the value is returned. The fourth argument -should point to an unsigned 32-bit integer. If no such value has been set, the -call to <b>pcre_fullinfo()</b> returns the error PCRE_ERROR_UNSET. -<pre> - PCRE_INFO_MAXLOOKBEHIND -</pre> -Return the number of characters (NB not data units) in the longest lookbehind -assertion in the pattern. This information is useful when doing multi-segment -matching using the partial matching facilities. Note that the simple assertions -\b and \B require a one-character lookbehind. \A also registers a -one-character lookbehind, though it does not actually inspect the previous -character. This is to ensure that at least one character from the old segment -is retained when a new segment is processed. Otherwise, if there are no -lookbehinds in the pattern, \A might match incorrectly at the start of a new -segment. -<pre> - PCRE_INFO_MINLENGTH -</pre> -If the pattern was studied and a minimum length for matching subject strings -was computed, its value is returned. Otherwise the returned value is -1. The -value is a number of characters, which in UTF mode may be different from the -number of data units. The fourth argument should point to an <b>int</b> -variable. A non-negative value is a lower bound to the length of any matching -string. There may not be any strings of that length that do actually match, but -every string that does match is at least that long. -<pre> - PCRE_INFO_NAMECOUNT - PCRE_INFO_NAMEENTRYSIZE - PCRE_INFO_NAMETABLE -</pre> -PCRE supports the use of named as well as numbered capturing parentheses. The -names are just an additional way of identifying the parentheses, which still -acquire numbers. Several convenience functions such as -<b>pcre_get_named_substring()</b> are provided for extracting captured -substrings by name. It is also possible to extract the data directly, by first -converting the name to a number in order to access the correct pointers in the -output vector (described with <b>pcre_exec()</b> below). To do the conversion, -you need to use the name-to-number map, which is described by these three -values. -</P> -<P> -The map consists of a number of fixed-size entries. PCRE_INFO_NAMECOUNT gives -the number of entries, and PCRE_INFO_NAMEENTRYSIZE gives the size of each -entry; both of these return an <b>int</b> value. The entry size depends on the -length of the longest name. PCRE_INFO_NAMETABLE returns a pointer to the first -entry of the table. This is a pointer to <b>char</b> in the 8-bit library, where -the first two bytes of each entry are the number of the capturing parenthesis, -most significant byte first. In the 16-bit library, the pointer points to -16-bit data units, the first of which contains the parenthesis number. In the -32-bit library, the pointer points to 32-bit data units, the first of which -contains the parenthesis number. The rest of the entry is the corresponding -name, zero terminated. -</P> -<P> -The names are in alphabetical order. If (?| is used to create multiple groups -with the same number, as described in the -<a href="pcrepattern.html#dupsubpatternnumber">section on duplicate subpattern numbers</a> -in the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -page, the groups may be given the same name, but there is only one entry in the -table. Different names for groups of the same number are not permitted. -Duplicate names for subpatterns with different numbers are permitted, -but only if PCRE_DUPNAMES is set. They appear in the table in the order in -which they were found in the pattern. In the absence of (?| this is the order -of increasing number; when (?| is used this is not necessarily the case because -later subpatterns may have lower numbers. -</P> -<P> -As a simple example of the name/number table, consider the following pattern -after compilation by the 8-bit library (assume PCRE_EXTENDED is set, so white -space - including newlines - is ignored): -<pre> - (?<date> (?<year>(\d\d)?\d\d) - (?<month>\d\d) - (?<day>\d\d) ) -</pre> -There are four named subpatterns, so the table has four entries, and each entry -in the table is eight bytes long. The table is as follows, with non-printing -bytes shows in hexadecimal, and undefined bytes shown as ??: -<pre> - 00 01 d a t e 00 ?? - 00 05 d a y 00 ?? ?? - 00 04 m o n t h 00 - 00 02 y e a r 00 ?? -</pre> -When writing code to extract data from named subpatterns using the -name-to-number map, remember that the length of the entries is likely to be -different for each compiled pattern. -<pre> - PCRE_INFO_OKPARTIAL -</pre> -Return 1 if the pattern can be used for partial matching with -<b>pcre_exec()</b>, otherwise 0. The fourth argument should point to an -<b>int</b> variable. From release 8.00, this always returns 1, because the -restrictions that previously applied to partial matching have been lifted. The -<a href="pcrepartial.html"><b>pcrepartial</b></a> -documentation gives details of partial matching. -<pre> - PCRE_INFO_OPTIONS -</pre> -Return a copy of the options with which the pattern was compiled. The fourth -argument should point to an <b>unsigned long int</b> variable. These option bits -are those specified in the call to <b>pcre_compile()</b>, modified by any -top-level option settings at the start of the pattern itself. In other words, -they are the options that will be in force when matching starts. For example, -if the pattern /(?im)abc(?-i)d/ is compiled with the PCRE_EXTENDED option, the -result is PCRE_CASELESS, PCRE_MULTILINE, and PCRE_EXTENDED. -</P> -<P> -A pattern is automatically anchored by PCRE if all of its top-level -alternatives begin with one of the following: -<pre> - ^ unless PCRE_MULTILINE is set - \A always - \G always - .* if PCRE_DOTALL is set and there are no back references to the subpattern in which .* appears -</pre> -For such patterns, the PCRE_ANCHORED bit is set in the options returned by -<b>pcre_fullinfo()</b>. -<pre> - PCRE_INFO_RECURSIONLIMIT -</pre> -If the pattern set a recursion limit by including an item of the form -(*LIMIT_RECURSION=nnnn) at the start, the value is returned. The fourth -argument should point to an unsigned 32-bit integer. If no such value has been -set, the call to <b>pcre_fullinfo()</b> returns the error PCRE_ERROR_UNSET. -<pre> - PCRE_INFO_SIZE -</pre> -Return the size of the compiled pattern in bytes (for all three libraries). The -fourth argument should point to a <b>size_t</b> variable. This value does not -include the size of the <b>pcre</b> structure that is returned by -<b>pcre_compile()</b>. The value that is passed as the argument to -<b>pcre_malloc()</b> when <b>pcre_compile()</b> is getting memory in which to -place the compiled data is the value returned by this option plus the size of -the <b>pcre</b> structure. Studying a compiled pattern, with or without JIT, -does not alter the value returned by this option. -<pre> - PCRE_INFO_STUDYSIZE -</pre> -Return the size in bytes (for all three libraries) of the data block pointed to -by the <i>study_data</i> field in a <b>pcre_extra</b> block. If <b>pcre_extra</b> -is NULL, or there is no study data, zero is returned. The fourth argument -should point to a <b>size_t</b> variable. The <i>study_data</i> field is set by -<b>pcre_study()</b> to record information that will speed up matching (see the -section entitled -<a href="#studyingapattern">"Studying a pattern"</a> -above). The format of the <i>study_data</i> block is private, but its length -is made available via this option so that it can be saved and restored (see the -<a href="pcreprecompile.html"><b>pcreprecompile</b></a> -documentation for details). -<pre> - PCRE_INFO_REQUIREDCHARFLAGS -</pre> -Returns 1 if there is a rightmost literal data unit that must exist in any -matched string, other than at its start. The fourth argument should point to -an <b>int</b> variable. If there is no such value, 0 is returned. If returning -1, the character value itself can be retrieved using PCRE_INFO_REQUIREDCHAR. -</P> -<P> -For anchored patterns, a last literal value is recorded only if it follows -something of variable length. For example, for the pattern /^a\d+z\d+/ the -returned value 1 (with "z" returned from PCRE_INFO_REQUIREDCHAR), but for -/^a\dz\d/ the returned value is 0. -<pre> - PCRE_INFO_REQUIREDCHAR -</pre> -Return the value of the rightmost literal data unit that must exist in any -matched string, other than at its start, if such a value has been recorded. The -fourth argument should point to an <b>uint32_t</b> variable. If there is no such -value, 0 is returned. -</P> -<br><a name="SEC16" href="#TOC1">REFERENCE COUNTS</a><br> -<P> -<b>int pcre_refcount(pcre *<i>code</i>, int <i>adjust</i>);</b> -</P> -<P> -The <b>pcre_refcount()</b> function is used to maintain a reference count in the -data block that contains a compiled pattern. It is provided for the benefit of -applications that operate in an object-oriented manner, where different parts -of the application may be using the same compiled pattern, but you want to free -the block when they are all done. -</P> -<P> -When a pattern is compiled, the reference count field is initialized to zero. -It is changed only by calling this function, whose action is to add the -<i>adjust</i> value (which may be positive or negative) to it. The yield of the -function is the new value. However, the value of the count is constrained to -lie between 0 and 65535, inclusive. If the new value is outside these limits, -it is forced to the appropriate limit value. -</P> -<P> -Except when it is zero, the reference count is not correctly preserved if a -pattern is compiled on one host and then transferred to a host whose byte-order -is different. (This seems a highly unlikely scenario.) -</P> -<br><a name="SEC17" href="#TOC1">MATCHING A PATTERN: THE TRADITIONAL FUNCTION</a><br> -<P> -<b>int pcre_exec(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> -<b> const char *<i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>);</b> -</P> -<P> -The function <b>pcre_exec()</b> is called to match a subject string against a -compiled pattern, which is passed in the <i>code</i> argument. If the -pattern was studied, the result of the study should be passed in the -<i>extra</i> argument. You can call <b>pcre_exec()</b> with the same <i>code</i> -and <i>extra</i> arguments as many times as you like, in order to match -different subject strings with the same pattern. -</P> -<P> -This function is the main matching facility of the library, and it operates in -a Perl-like manner. For specialist use there is also an alternative matching -function, which is described -<a href="#dfamatch">below</a> -in the section about the <b>pcre_dfa_exec()</b> function. -</P> -<P> -In most applications, the pattern will have been compiled (and optionally -studied) in the same process that calls <b>pcre_exec()</b>. However, it is -possible to save compiled patterns and study data, and then use them later -in different processes, possibly even on different hosts. For a discussion -about this, see the -<a href="pcreprecompile.html"><b>pcreprecompile</b></a> -documentation. -</P> -<P> -Here is an example of a simple call to <b>pcre_exec()</b>: -<pre> - int rc; - int ovector[30]; - rc = pcre_exec( - re, /* result of pcre_compile() */ - NULL, /* we didn't study the pattern */ - "some string", /* the subject string */ - 11, /* the length of the subject string */ - 0, /* start at offset 0 in the subject */ - 0, /* default options */ - ovector, /* vector of integers for substring information */ - 30); /* number of elements (NOT size in bytes) */ -<a name="extradata"></a></PRE> -</P> -<br><b> -Extra data for <b>pcre_exec()</b> -</b><br> -<P> -If the <i>extra</i> argument is not NULL, it must point to a <b>pcre_extra</b> -data block. The <b>pcre_study()</b> function returns such a block (when it -doesn't return NULL), but you can also create one for yourself, and pass -additional information in it. The <b>pcre_extra</b> block contains the following -fields (not necessarily in this order): -<pre> - unsigned long int <i>flags</i>; - void *<i>study_data</i>; - void *<i>executable_jit</i>; - unsigned long int <i>match_limit</i>; - unsigned long int <i>match_limit_recursion</i>; - void *<i>callout_data</i>; - const unsigned char *<i>tables</i>; - unsigned char **<i>mark</i>; -</pre> -In the 16-bit version of this structure, the <i>mark</i> field has type -"PCRE_UCHAR16 **". -<br> -<br> -In the 32-bit version of this structure, the <i>mark</i> field has type -"PCRE_UCHAR32 **". -</P> -<P> -The <i>flags</i> field is used to specify which of the other fields are set. The -flag bits are: -<pre> - PCRE_EXTRA_CALLOUT_DATA - PCRE_EXTRA_EXECUTABLE_JIT - PCRE_EXTRA_MARK - PCRE_EXTRA_MATCH_LIMIT - PCRE_EXTRA_MATCH_LIMIT_RECURSION - PCRE_EXTRA_STUDY_DATA - PCRE_EXTRA_TABLES -</pre> -Other flag bits should be set to zero. The <i>study_data</i> field and sometimes -the <i>executable_jit</i> field are set in the <b>pcre_extra</b> block that is -returned by <b>pcre_study()</b>, together with the appropriate flag bits. You -should not set these yourself, but you may add to the block by setting other -fields and their corresponding flag bits. -</P> -<P> -The <i>match_limit</i> field provides a means of preventing PCRE from using up a -vast amount of resources when running patterns that are not going to match, -but which have a very large number of possibilities in their search trees. The -classic example is a pattern that uses nested unlimited repeats. -</P> -<P> -Internally, <b>pcre_exec()</b> uses a function called <b>match()</b>, which it -calls repeatedly (sometimes recursively). The limit set by <i>match_limit</i> is -imposed on the number of times this function is called during a match, which -has the effect of limiting the amount of backtracking that can take place. For -patterns that are not anchored, the count restarts from zero for each position -in the subject string. -</P> -<P> -When <b>pcre_exec()</b> is called with a pattern that was successfully studied -with a JIT option, the way that the matching is executed is entirely different. -However, there is still the possibility of runaway matching that goes on for a -very long time, and so the <i>match_limit</i> value is also used in this case -(but in a different way) to limit how long the matching can continue. -</P> -<P> -The default value for the limit can be set when PCRE is built; the default -default is 10 million, which handles all but the most extreme cases. You can -override the default by suppling <b>pcre_exec()</b> with a <b>pcre_extra</b> -block in which <i>match_limit</i> is set, and PCRE_EXTRA_MATCH_LIMIT is set in -the <i>flags</i> field. If the limit is exceeded, <b>pcre_exec()</b> returns -PCRE_ERROR_MATCHLIMIT. -</P> -<P> -A value for the match limit may also be supplied by an item at the start of a -pattern of the form -<pre> - (*LIMIT_MATCH=d) -</pre> -where d is a decimal number. However, such a setting is ignored unless d is -less than the limit set by the caller of <b>pcre_exec()</b> or, if no such limit -is set, less than the default. -</P> -<P> -The <i>match_limit_recursion</i> field is similar to <i>match_limit</i>, but -instead of limiting the total number of times that <b>match()</b> is called, it -limits the depth of recursion. The recursion depth is a smaller number than the -total number of calls, because not all calls to <b>match()</b> are recursive. -This limit is of use only if it is set smaller than <i>match_limit</i>. -</P> -<P> -Limiting the recursion depth limits the amount of machine stack that can be -used, or, when PCRE has been compiled to use memory on the heap instead of the -stack, the amount of heap memory that can be used. This limit is not relevant, -and is ignored, when matching is done using JIT compiled code. -</P> -<P> -The default value for <i>match_limit_recursion</i> can be set when PCRE is -built; the default default is the same value as the default for -<i>match_limit</i>. You can override the default by suppling <b>pcre_exec()</b> -with a <b>pcre_extra</b> block in which <i>match_limit_recursion</i> is set, and -PCRE_EXTRA_MATCH_LIMIT_RECURSION is set in the <i>flags</i> field. If the limit -is exceeded, <b>pcre_exec()</b> returns PCRE_ERROR_RECURSIONLIMIT. -</P> -<P> -A value for the recursion limit may also be supplied by an item at the start of -a pattern of the form -<pre> - (*LIMIT_RECURSION=d) -</pre> -where d is a decimal number. However, such a setting is ignored unless d is -less than the limit set by the caller of <b>pcre_exec()</b> or, if no such limit -is set, less than the default. -</P> -<P> -The <i>callout_data</i> field is used in conjunction with the "callout" feature, -and is described in the -<a href="pcrecallout.html"><b>pcrecallout</b></a> -documentation. -</P> -<P> -The <i>tables</i> field is provided for use with patterns that have been -pre-compiled using custom character tables, saved to disc or elsewhere, and -then reloaded, because the tables that were used to compile a pattern are not -saved with it. See the -<a href="pcreprecompile.html"><b>pcreprecompile</b></a> -documentation for a discussion of saving compiled patterns for later use. If -NULL is passed using this mechanism, it forces PCRE's internal tables to be -used. -</P> -<P> -<b>Warning:</b> The tables that <b>pcre_exec()</b> uses must be the same as those -that were used when the pattern was compiled. If this is not the case, the -behaviour of <b>pcre_exec()</b> is undefined. Therefore, when a pattern is -compiled and matched in the same process, this field should never be set. In -this (the most common) case, the correct table pointer is automatically passed -with the compiled pattern from <b>pcre_compile()</b> to <b>pcre_exec()</b>. -</P> -<P> -If PCRE_EXTRA_MARK is set in the <i>flags</i> field, the <i>mark</i> field must -be set to point to a suitable variable. If the pattern contains any -backtracking control verbs such as (*MARK:NAME), and the execution ends up with -a name to pass back, a pointer to the name string (zero terminated) is placed -in the variable pointed to by the <i>mark</i> field. The names are within the -compiled pattern; if you wish to retain such a name you must copy it before -freeing the memory of a compiled pattern. If there is no name to pass back, the -variable pointed to by the <i>mark</i> field is set to NULL. For details of the -backtracking control verbs, see the section entitled -<a href="pcrepattern#backtrackcontrol">"Backtracking control"</a> -in the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -documentation. -<a name="execoptions"></a></P> -<br><b> -Option bits for <b>pcre_exec()</b> -</b><br> -<P> -The unused bits of the <i>options</i> argument for <b>pcre_exec()</b> must be -zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_<i>xxx</i>, -PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, -PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_HARD, and -PCRE_PARTIAL_SOFT. -</P> -<P> -If the pattern was successfully studied with one of the just-in-time (JIT) -compile options, the only supported options for JIT execution are -PCRE_NO_UTF8_CHECK, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, -PCRE_NOTEMPTY_ATSTART, PCRE_PARTIAL_HARD, and PCRE_PARTIAL_SOFT. If an -unsupported option is used, JIT execution is disabled and the normal -interpretive code in <b>pcre_exec()</b> is run. -<pre> - PCRE_ANCHORED -</pre> -The PCRE_ANCHORED option limits <b>pcre_exec()</b> to matching at the first -matching position. If a pattern was compiled with PCRE_ANCHORED, or turned out -to be anchored by virtue of its contents, it cannot be made unachored at -matching time. -<pre> - PCRE_BSR_ANYCRLF - PCRE_BSR_UNICODE -</pre> -These options (which are mutually exclusive) control what the \R escape -sequence matches. The choice is either to match only CR, LF, or CRLF, or to -match any Unicode newline sequence. These options override the choice that was -made or defaulted when the pattern was compiled. -<pre> - PCRE_NEWLINE_CR - PCRE_NEWLINE_LF - PCRE_NEWLINE_CRLF - PCRE_NEWLINE_ANYCRLF - PCRE_NEWLINE_ANY -</pre> -These options override the newline definition that was chosen or defaulted when -the pattern was compiled. For details, see the description of -<b>pcre_compile()</b> above. During matching, the newline choice affects the -behaviour of the dot, circumflex, and dollar metacharacters. It may also alter -the way the match position is advanced after a match failure for an unanchored -pattern. -</P> -<P> -When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is set, and a -match attempt for an unanchored pattern fails when the current position is at a -CRLF sequence, and the pattern contains no explicit matches for CR or LF -characters, the match position is advanced by two characters instead of one, in -other words, to after the CRLF. -</P> -<P> -The above rule is a compromise that makes the most common cases work as -expected. For example, if the pattern is .+A (and the PCRE_DOTALL option is not -set), it does not match the string "\r\nA" because, after failing at the -start, it skips both the CR and the LF before retrying. However, the pattern -[\r\n]A does match that string, because it contains an explicit CR or LF -reference, and so advances only by one character after the first failure. -</P> -<P> -An explicit match for CR of LF is either a literal appearance of one of those -characters, or one of the \r or \n escape sequences. Implicit matches such as -[^X] do not count, nor does \s (which includes CR and LF in the characters -that it matches). -</P> -<P> -Notwithstanding the above, anomalous effects may still occur when CRLF is a -valid newline sequence and explicit \r or \n escapes appear in the pattern. -<pre> - PCRE_NOTBOL -</pre> -This option specifies that first character of the subject string is not the -beginning of a line, so the circumflex metacharacter should not match before -it. Setting this without PCRE_MULTILINE (at compile time) causes circumflex -never to match. This option affects only the behaviour of the circumflex -metacharacter. It does not affect \A. -<pre> - PCRE_NOTEOL -</pre> -This option specifies that the end of the subject string is not the end of a -line, so the dollar metacharacter should not match it nor (except in multiline -mode) a newline immediately before it. Setting this without PCRE_MULTILINE (at -compile time) causes dollar never to match. This option affects only the -behaviour of the dollar metacharacter. It does not affect \Z or \z. -<pre> - PCRE_NOTEMPTY -</pre> -An empty string is not considered to be a valid match if this option is set. If -there are alternatives in the pattern, they are tried. If all the alternatives -match the empty string, the entire match fails. For example, if the pattern -<pre> - a?b? -</pre> -is applied to a string not beginning with "a" or "b", it matches an empty -string at the start of the subject. With PCRE_NOTEMPTY set, this match is not -valid, so PCRE searches further into the string for occurrences of "a" or "b". -<pre> - PCRE_NOTEMPTY_ATSTART -</pre> -This is like PCRE_NOTEMPTY, except that an empty string match that is not at -the start of the subject is permitted. If the pattern is anchored, such a match -can occur only if the pattern contains \K. -</P> -<P> -Perl has no direct equivalent of PCRE_NOTEMPTY or PCRE_NOTEMPTY_ATSTART, but it -does make a special case of a pattern match of the empty string within its -<b>split()</b> function, and when using the /g modifier. It is possible to -emulate Perl's behaviour after matching a null string by first trying the match -again at the same offset with PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED, and then -if that fails, by advancing the starting offset (see below) and trying an -ordinary match again. There is some code that demonstrates how to do this in -the -<a href="pcredemo.html"><b>pcredemo</b></a> -sample program. In the most general case, you have to check to see if the -newline convention recognizes CRLF as a newline, and if so, and the current -character is CR followed by LF, advance the starting offset by two characters -instead of one. -<pre> - PCRE_NO_START_OPTIMIZE -</pre> -There are a number of optimizations that <b>pcre_exec()</b> uses at the start of -a match, in order to speed up the process. For example, if it is known that an -unanchored match must start with a specific character, it searches the subject -for that character, and fails immediately if it cannot find it, without -actually running the main matching function. This means that a special item -such as (*COMMIT) at the start of a pattern is not considered until after a -suitable starting point for the match has been found. Also, when callouts or -(*MARK) items are in use, these "start-up" optimizations can cause them to be -skipped if the pattern is never actually used. The start-up optimizations are -in effect a pre-scan of the subject that takes place before the pattern is run. -</P> -<P> -The PCRE_NO_START_OPTIMIZE option disables the start-up optimizations, possibly -causing performance to suffer, but ensuring that in cases where the result is -"no match", the callouts do occur, and that items such as (*COMMIT) and (*MARK) -are considered at every possible starting position in the subject string. If -PCRE_NO_START_OPTIMIZE is set at compile time, it cannot be unset at matching -time. The use of PCRE_NO_START_OPTIMIZE at matching time (that is, passing it -to <b>pcre_exec()</b>) disables JIT execution; in this situation, matching is -always done using interpretively. -</P> -<P> -Setting PCRE_NO_START_OPTIMIZE can change the outcome of a matching operation. -Consider the pattern -<pre> - (*COMMIT)ABC -</pre> -When this is compiled, PCRE records the fact that a match must start with the -character "A". Suppose the subject string is "DEFABC". The start-up -optimization scans along the subject, finds "A" and runs the first match -attempt from there. The (*COMMIT) item means that the pattern must match the -current starting position, which in this case, it does. However, if the same -match is run with PCRE_NO_START_OPTIMIZE set, the initial scan along the -subject string does not happen. The first match attempt is run starting from -"D" and when this fails, (*COMMIT) prevents any further matches being tried, so -the overall result is "no match". If the pattern is studied, more start-up -optimizations may be used. For example, a minimum length for the subject may be -recorded. Consider the pattern -<pre> - (*MARK:A)(X|Y) -</pre> -The minimum length for a match is one character. If the subject is "ABC", there -will be attempts to match "ABC", "BC", "C", and then finally an empty string. -If the pattern is studied, the final attempt does not take place, because PCRE -knows that the subject is too short, and so the (*MARK) is never encountered. -In this case, studying the pattern does not affect the overall match result, -which is still "no match", but it does affect the auxiliary information that is -returned. -<pre> - PCRE_NO_UTF8_CHECK -</pre> -When PCRE_UTF8 is set at compile time, the validity of the subject as a UTF-8 -string is automatically checked when <b>pcre_exec()</b> is subsequently called. -The entire string is checked before any other processing takes place. The value -of <i>startoffset</i> is also checked to ensure that it points to the start of a -UTF-8 character. There is a discussion about the -<a href="pcreunicode.html#utf8strings">validity of UTF-8 strings</a> -in the -<a href="pcreunicode.html"><b>pcreunicode</b></a> -page. If an invalid sequence of bytes is found, <b>pcre_exec()</b> returns the -error PCRE_ERROR_BADUTF8 or, if PCRE_PARTIAL_HARD is set and the problem is a -truncated character at the end of the subject, PCRE_ERROR_SHORTUTF8. In both -cases, information about the precise nature of the error may also be returned -(see the descriptions of these errors in the section entitled \fIError return -values from\fP <b>pcre_exec()</b> -<a href="#errorlist">below).</a> -If <i>startoffset</i> contains a value that does not point to the start of a -UTF-8 character (or to the end of the subject), PCRE_ERROR_BADUTF8_OFFSET is -returned. -</P> -<P> -If you already know that your subject is valid, and you want to skip these -checks for performance reasons, you can set the PCRE_NO_UTF8_CHECK option when -calling <b>pcre_exec()</b>. You might want to do this for the second and -subsequent calls to <b>pcre_exec()</b> if you are making repeated calls to find -all the matches in a single subject string. However, you should be sure that -the value of <i>startoffset</i> points to the start of a character (or the end -of the subject). When PCRE_NO_UTF8_CHECK is set, the effect of passing an -invalid string as a subject or an invalid value of <i>startoffset</i> is -undefined. Your program may crash or loop. -<pre> - PCRE_PARTIAL_HARD - PCRE_PARTIAL_SOFT -</pre> -These options turn on the partial matching feature. For backwards -compatibility, PCRE_PARTIAL is a synonym for PCRE_PARTIAL_SOFT. A partial match -occurs if the end of the subject string is reached successfully, but there are -not enough subject characters to complete the match. If this happens when -PCRE_PARTIAL_SOFT (but not PCRE_PARTIAL_HARD) is set, matching continues by -testing any remaining alternatives. Only if no complete match can be found is -PCRE_ERROR_PARTIAL returned instead of PCRE_ERROR_NOMATCH. In other words, -PCRE_PARTIAL_SOFT says that the caller is prepared to handle a partial match, -but only if no complete match can be found. -</P> -<P> -If PCRE_PARTIAL_HARD is set, it overrides PCRE_PARTIAL_SOFT. In this case, if a -partial match is found, <b>pcre_exec()</b> immediately returns -PCRE_ERROR_PARTIAL, without considering any other alternatives. In other words, -when PCRE_PARTIAL_HARD is set, a partial match is considered to be more -important that an alternative complete match. -</P> -<P> -In both cases, the portion of the string that was inspected when the partial -match was found is set as the first matching string. There is a more detailed -discussion of partial and multi-segment matching, with examples, in the -<a href="pcrepartial.html"><b>pcrepartial</b></a> -documentation. -</P> -<br><b> -The string to be matched by <b>pcre_exec()</b> -</b><br> -<P> -The subject string is passed to <b>pcre_exec()</b> as a pointer in -<i>subject</i>, a length in <i>length</i>, and a starting offset in -<i>startoffset</i>. The units for <i>length</i> and <i>startoffset</i> are bytes -for the 8-bit library, 16-bit data items for the 16-bit library, and 32-bit -data items for the 32-bit library. -</P> -<P> -If <i>startoffset</i> is negative or greater than the length of the subject, -<b>pcre_exec()</b> returns PCRE_ERROR_BADOFFSET. When the starting offset is -zero, the search for a match starts at the beginning of the subject, and this -is by far the most common case. In UTF-8 or UTF-16 mode, the offset must point -to the start of a character, or the end of the subject (in UTF-32 mode, one -data unit equals one character, so all offsets are valid). Unlike the pattern -string, the subject may contain binary zeroes. -</P> -<P> -A non-zero starting offset is useful when searching for another match in the -same subject by calling <b>pcre_exec()</b> again after a previous success. -Setting <i>startoffset</i> differs from just passing over a shortened string and -setting PCRE_NOTBOL in the case of a pattern that begins with any kind of -lookbehind. For example, consider the pattern -<pre> - \Biss\B -</pre> -which finds occurrences of "iss" in the middle of words. (\B matches only if -the current position in the subject is not a word boundary.) When applied to -the string "Mississipi" the first call to <b>pcre_exec()</b> finds the first -occurrence. If <b>pcre_exec()</b> is called again with just the remainder of the -subject, namely "issipi", it does not match, because \B is always false at the -start of the subject, which is deemed to be a word boundary. However, if -<b>pcre_exec()</b> is passed the entire string again, but with <i>startoffset</i> -set to 4, it finds the second occurrence of "iss" because it is able to look -behind the starting point to discover that it is preceded by a letter. -</P> -<P> -Finding all the matches in a subject is tricky when the pattern can match an -empty string. It is possible to emulate Perl's /g behaviour by first trying the -match again at the same offset, with the PCRE_NOTEMPTY_ATSTART and -PCRE_ANCHORED options, and then if that fails, advancing the starting offset -and trying an ordinary match again. There is some code that demonstrates how to -do this in the -<a href="pcredemo.html"><b>pcredemo</b></a> -sample program. In the most general case, you have to check to see if the -newline convention recognizes CRLF as a newline, and if so, and the current -character is CR followed by LF, advance the starting offset by two characters -instead of one. -</P> -<P> -If a non-zero starting offset is passed when the pattern is anchored, one -attempt to match at the given offset is made. This can only succeed if the -pattern does not require the match to be at the start of the subject. -</P> -<br><b> -How <b>pcre_exec()</b> returns captured substrings -</b><br> -<P> -In general, a pattern matches a certain portion of the subject, and in -addition, further substrings from the subject may be picked out by parts of the -pattern. Following the usage in Jeffrey Friedl's book, this is called -"capturing" in what follows, and the phrase "capturing subpattern" is used for -a fragment of a pattern that picks out a substring. PCRE supports several other -kinds of parenthesized subpattern that do not cause substrings to be captured. -</P> -<P> -Captured substrings are returned to the caller via a vector of integers whose -address is passed in <i>ovector</i>. The number of elements in the vector is -passed in <i>ovecsize</i>, which must be a non-negative number. <b>Note</b>: this -argument is NOT the size of <i>ovector</i> in bytes. -</P> -<P> -The first two-thirds of the vector is used to pass back captured substrings, -each substring using a pair of integers. The remaining third of the vector is -used as workspace by <b>pcre_exec()</b> while matching capturing subpatterns, -and is not available for passing back information. The number passed in -<i>ovecsize</i> should always be a multiple of three. If it is not, it is -rounded down. -</P> -<P> -When a match is successful, information about captured substrings is returned -in pairs of integers, starting at the beginning of <i>ovector</i>, and -continuing up to two-thirds of its length at the most. The first element of -each pair is set to the offset of the first character in a substring, and the -second is set to the offset of the first character after the end of a -substring. These values are always data unit offsets, even in UTF mode. They -are byte offsets in the 8-bit library, 16-bit data item offsets in the 16-bit -library, and 32-bit data item offsets in the 32-bit library. <b>Note</b>: they -are not character counts. -</P> -<P> -The first pair of integers, <i>ovector[0]</i> and <i>ovector[1]</i>, identify the -portion of the subject string matched by the entire pattern. The next pair is -used for the first capturing subpattern, and so on. The value returned by -<b>pcre_exec()</b> is one more than the highest numbered pair that has been set. -For example, if two substrings have been captured, the returned value is 3. If -there are no capturing subpatterns, the return value from a successful match is -1, indicating that just the first pair of offsets has been set. -</P> -<P> -If a capturing subpattern is matched repeatedly, it is the last portion of the -string that it matched that is returned. -</P> -<P> -If the vector is too small to hold all the captured substring offsets, it is -used as far as possible (up to two-thirds of its length), and the function -returns a value of zero. If neither the actual string matched nor any captured -substrings are of interest, <b>pcre_exec()</b> may be called with <i>ovector</i> -passed as NULL and <i>ovecsize</i> as zero. However, if the pattern contains -back references and the <i>ovector</i> is not big enough to remember the related -substrings, PCRE has to get additional memory for use during matching. Thus it -is usually advisable to supply an <i>ovector</i> of reasonable size. -</P> -<P> -There are some cases where zero is returned (indicating vector overflow) when -in fact the vector is exactly the right size for the final match. For example, -consider the pattern -<pre> - (a)(?:(b)c|bd) -</pre> -If a vector of 6 elements (allowing for only 1 captured substring) is given -with subject string "abd", <b>pcre_exec()</b> will try to set the second -captured string, thereby recording a vector overflow, before failing to match -"c" and backing up to try the second alternative. The zero return, however, -does correctly indicate that the maximum number of slots (namely 2) have been -filled. In similar cases where there is temporary overflow, but the final -number of used slots is actually less than the maximum, a non-zero value is -returned. -</P> -<P> -The <b>pcre_fullinfo()</b> function can be used to find out how many capturing -subpatterns there are in a compiled pattern. The smallest size for -<i>ovector</i> that will allow for <i>n</i> captured substrings, in addition to -the offsets of the substring matched by the whole pattern, is (<i>n</i>+1)*3. -</P> -<P> -It is possible for capturing subpattern number <i>n+1</i> to match some part of -the subject when subpattern <i>n</i> has not been used at all. For example, if -the string "abc" is matched against the pattern (a|(z))(bc) the return from the -function is 4, and subpatterns 1 and 3 are matched, but 2 is not. When this -happens, both values in the offset pairs corresponding to unused subpatterns -are set to -1. -</P> -<P> -Offset values that correspond to unused subpatterns at the end of the -expression are also set to -1. For example, if the string "abc" is matched -against the pattern (abc)(x(yz)?)? subpatterns 2 and 3 are not matched. The -return from the function is 2, because the highest used capturing subpattern -number is 1, and the offsets for for the second and third capturing subpatterns -(assuming the vector is large enough, of course) are set to -1. -</P> -<P> -<b>Note</b>: Elements in the first two-thirds of <i>ovector</i> that do not -correspond to capturing parentheses in the pattern are never changed. That is, -if a pattern contains <i>n</i> capturing parentheses, no more than -<i>ovector[0]</i> to <i>ovector[2n+1]</i> are set by <b>pcre_exec()</b>. The other -elements (in the first two-thirds) retain whatever values they previously had. -</P> -<P> -Some convenience functions are provided for extracting the captured substrings -as separate strings. These are described below. -<a name="errorlist"></a></P> -<br><b> -Error return values from <b>pcre_exec()</b> -</b><br> -<P> -If <b>pcre_exec()</b> fails, it returns a negative number. The following are -defined in the header file: -<pre> - PCRE_ERROR_NOMATCH (-1) -</pre> -The subject string did not match the pattern. -<pre> - PCRE_ERROR_NULL (-2) -</pre> -Either <i>code</i> or <i>subject</i> was passed as NULL, or <i>ovector</i> was -NULL and <i>ovecsize</i> was not zero. -<pre> - PCRE_ERROR_BADOPTION (-3) -</pre> -An unrecognized bit was set in the <i>options</i> argument. -<pre> - PCRE_ERROR_BADMAGIC (-4) -</pre> -PCRE stores a 4-byte "magic number" at the start of the compiled code, to catch -the case when it is passed a junk pointer and to detect when a pattern that was -compiled in an environment of one endianness is run in an environment with the -other endianness. This is the error that PCRE gives when the magic number is -not present. -<pre> - PCRE_ERROR_UNKNOWN_OPCODE (-5) -</pre> -While running the pattern match, an unknown item was encountered in the -compiled pattern. This error could be caused by a bug in PCRE or by overwriting -of the compiled pattern. -<pre> - PCRE_ERROR_NOMEMORY (-6) -</pre> -If a pattern contains back references, but the <i>ovector</i> that is passed to -<b>pcre_exec()</b> is not big enough to remember the referenced substrings, PCRE -gets a block of memory at the start of matching to use for this purpose. If the -call via <b>pcre_malloc()</b> fails, this error is given. The memory is -automatically freed at the end of matching. -</P> -<P> -This error is also given if <b>pcre_stack_malloc()</b> fails in -<b>pcre_exec()</b>. This can happen only when PCRE has been compiled with -<b>--disable-stack-for-recursion</b>. -<pre> - PCRE_ERROR_NOSUBSTRING (-7) -</pre> -This error is used by the <b>pcre_copy_substring()</b>, -<b>pcre_get_substring()</b>, and <b>pcre_get_substring_list()</b> functions (see -below). It is never returned by <b>pcre_exec()</b>. -<pre> - PCRE_ERROR_MATCHLIMIT (-8) -</pre> -The backtracking limit, as specified by the <i>match_limit</i> field in a -<b>pcre_extra</b> structure (or defaulted) was reached. See the description -above. -<pre> - PCRE_ERROR_CALLOUT (-9) -</pre> -This error is never generated by <b>pcre_exec()</b> itself. It is provided for -use by callout functions that want to yield a distinctive error code. See the -<a href="pcrecallout.html"><b>pcrecallout</b></a> -documentation for details. -<pre> - PCRE_ERROR_BADUTF8 (-10) -</pre> -A string that contains an invalid UTF-8 byte sequence was passed as a subject, -and the PCRE_NO_UTF8_CHECK option was not set. If the size of the output vector -(<i>ovecsize</i>) is at least 2, the byte offset to the start of the the invalid -UTF-8 character is placed in the first element, and a reason code is placed in -the second element. The reason codes are listed in the -<a href="#badutf8reasons">following section.</a> -For backward compatibility, if PCRE_PARTIAL_HARD is set and the problem is a -truncated UTF-8 character at the end of the subject (reason codes 1 to 5), -PCRE_ERROR_SHORTUTF8 is returned instead of PCRE_ERROR_BADUTF8. -<pre> - PCRE_ERROR_BADUTF8_OFFSET (-11) -</pre> -The UTF-8 byte sequence that was passed as a subject was checked and found to -be valid (the PCRE_NO_UTF8_CHECK option was not set), but the value of -<i>startoffset</i> did not point to the beginning of a UTF-8 character or the -end of the subject. -<pre> - PCRE_ERROR_PARTIAL (-12) -</pre> -The subject string did not match, but it did match partially. See the -<a href="pcrepartial.html"><b>pcrepartial</b></a> -documentation for details of partial matching. -<pre> - PCRE_ERROR_BADPARTIAL (-13) -</pre> -This code is no longer in use. It was formerly returned when the PCRE_PARTIAL -option was used with a compiled pattern containing items that were not -supported for partial matching. From release 8.00 onwards, there are no -restrictions on partial matching. -<pre> - PCRE_ERROR_INTERNAL (-14) -</pre> -An unexpected internal error has occurred. This error could be caused by a bug -in PCRE or by overwriting of the compiled pattern. -<pre> - PCRE_ERROR_BADCOUNT (-15) -</pre> -This error is given if the value of the <i>ovecsize</i> argument is negative. -<pre> - PCRE_ERROR_RECURSIONLIMIT (-21) -</pre> -The internal recursion limit, as specified by the <i>match_limit_recursion</i> -field in a <b>pcre_extra</b> structure (or defaulted) was reached. See the -description above. -<pre> - PCRE_ERROR_BADNEWLINE (-23) -</pre> -An invalid combination of PCRE_NEWLINE_<i>xxx</i> options was given. -<pre> - PCRE_ERROR_BADOFFSET (-24) -</pre> -The value of <i>startoffset</i> was negative or greater than the length of the -subject, that is, the value in <i>length</i>. -<pre> - PCRE_ERROR_SHORTUTF8 (-25) -</pre> -This error is returned instead of PCRE_ERROR_BADUTF8 when the subject string -ends with a truncated UTF-8 character and the PCRE_PARTIAL_HARD option is set. -Information about the failure is returned as for PCRE_ERROR_BADUTF8. It is in -fact sufficient to detect this case, but this special error code for -PCRE_PARTIAL_HARD precedes the implementation of returned information; it is -retained for backwards compatibility. -<pre> - PCRE_ERROR_RECURSELOOP (-26) -</pre> -This error is returned when <b>pcre_exec()</b> detects a recursion loop within -the pattern. Specifically, it means that either the whole pattern or a -subpattern has been called recursively for the second time at the same position -in the subject string. Some simple patterns that might do this are detected and -faulted at compile time, but more complicated cases, in particular mutual -recursions between two different subpatterns, cannot be detected until run -time. -<pre> - PCRE_ERROR_JIT_STACKLIMIT (-27) -</pre> -This error is returned when a pattern that was successfully studied using a -JIT compile option is being matched, but the memory available for the -just-in-time processing stack is not large enough. See the -<a href="pcrejit.html"><b>pcrejit</b></a> -documentation for more details. -<pre> - PCRE_ERROR_BADMODE (-28) -</pre> -This error is given if a pattern that was compiled by the 8-bit library is -passed to a 16-bit or 32-bit library function, or vice versa. -<pre> - PCRE_ERROR_BADENDIANNESS (-29) -</pre> -This error is given if a pattern that was compiled and saved is reloaded on a -host with different endianness. The utility function -<b>pcre_pattern_to_host_byte_order()</b> can be used to convert such a pattern -so that it runs on the new host. -<pre> - PCRE_ERROR_JIT_BADOPTION -</pre> -This error is returned when a pattern that was successfully studied using a JIT -compile option is being matched, but the matching mode (partial or complete -match) does not correspond to any JIT compilation mode. When the JIT fast path -function is used, this error may be also given for invalid options. See the -<a href="pcrejit.html"><b>pcrejit</b></a> -documentation for more details. -<pre> - PCRE_ERROR_BADLENGTH (-32) -</pre> -This error is given if <b>pcre_exec()</b> is called with a negative value for -the <i>length</i> argument. -</P> -<P> -Error numbers -16 to -20, -22, and 30 are not used by <b>pcre_exec()</b>. -<a name="badutf8reasons"></a></P> -<br><b> -Reason codes for invalid UTF-8 strings -</b><br> -<P> -This section applies only to the 8-bit library. The corresponding information -for the 16-bit and 32-bit libraries is given in the -<a href="pcre16.html"><b>pcre16</b></a> -and -<a href="pcre32.html"><b>pcre32</b></a> -pages. -</P> -<P> -When <b>pcre_exec()</b> returns either PCRE_ERROR_BADUTF8 or -PCRE_ERROR_SHORTUTF8, and the size of the output vector (<i>ovecsize</i>) is at -least 2, the offset of the start of the invalid UTF-8 character is placed in -the first output vector element (<i>ovector[0]</i>) and a reason code is placed -in the second element (<i>ovector[1]</i>). The reason codes are given names in -the <b>pcre.h</b> header file: -<pre> - PCRE_UTF8_ERR1 - PCRE_UTF8_ERR2 - PCRE_UTF8_ERR3 - PCRE_UTF8_ERR4 - PCRE_UTF8_ERR5 -</pre> -The string ends with a truncated UTF-8 character; the code specifies how many -bytes are missing (1 to 5). Although RFC 3629 restricts UTF-8 characters to be -no longer than 4 bytes, the encoding scheme (originally defined by RFC 2279) -allows for up to 6 bytes, and this is checked first; hence the possibility of -4 or 5 missing bytes. -<pre> - PCRE_UTF8_ERR6 - PCRE_UTF8_ERR7 - PCRE_UTF8_ERR8 - PCRE_UTF8_ERR9 - PCRE_UTF8_ERR10 -</pre> -The two most significant bits of the 2nd, 3rd, 4th, 5th, or 6th byte of the -character do not have the binary value 0b10 (that is, either the most -significant bit is 0, or the next bit is 1). -<pre> - PCRE_UTF8_ERR11 - PCRE_UTF8_ERR12 -</pre> -A character that is valid by the RFC 2279 rules is either 5 or 6 bytes long; -these code points are excluded by RFC 3629. -<pre> - PCRE_UTF8_ERR13 -</pre> -A 4-byte character has a value greater than 0x10fff; these code points are -excluded by RFC 3629. -<pre> - PCRE_UTF8_ERR14 -</pre> -A 3-byte character has a value in the range 0xd800 to 0xdfff; this range of -code points are reserved by RFC 3629 for use with UTF-16, and so are excluded -from UTF-8. -<pre> - PCRE_UTF8_ERR15 - PCRE_UTF8_ERR16 - PCRE_UTF8_ERR17 - PCRE_UTF8_ERR18 - PCRE_UTF8_ERR19 -</pre> -A 2-, 3-, 4-, 5-, or 6-byte character is "overlong", that is, it codes for a -value that can be represented by fewer bytes, which is invalid. For example, -the two bytes 0xc0, 0xae give the value 0x2e, whose correct coding uses just -one byte. -<pre> - PCRE_UTF8_ERR20 -</pre> -The two most significant bits of the first byte of a character have the binary -value 0b10 (that is, the most significant bit is 1 and the second is 0). Such a -byte can only validly occur as the second or subsequent byte of a multi-byte -character. -<pre> - PCRE_UTF8_ERR21 -</pre> -The first byte of a character has the value 0xfe or 0xff. These values can -never occur in a valid UTF-8 string. -<pre> - PCRE_UTF8_ERR22 -</pre> -This error code was formerly used when the presence of a so-called -"non-character" caused an error. Unicode corrigendum #9 makes it clear that -such characters should not cause a string to be rejected, and so this code is -no longer in use and is never returned. -</P> -<br><a name="SEC18" href="#TOC1">EXTRACTING CAPTURED SUBSTRINGS BY NUMBER</a><br> -<P> -<b>int pcre_copy_substring(const char *<i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>, char *<i>buffer</i>,</b> -<b> int <i>buffersize</i>);</b> -<br> -<br> -<b>int pcre_get_substring(const char *<i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, int <i>stringnumber</i>,</b> -<b> const char **<i>stringptr</i>);</b> -<br> -<br> -<b>int pcre_get_substring_list(const char *<i>subject</i>,</b> -<b> int *<i>ovector</i>, int <i>stringcount</i>, const char ***<i>listptr</i>);</b> -</P> -<P> -Captured substrings can be accessed directly by using the offsets returned by -<b>pcre_exec()</b> in <i>ovector</i>. For convenience, the functions -<b>pcre_copy_substring()</b>, <b>pcre_get_substring()</b>, and -<b>pcre_get_substring_list()</b> are provided for extracting captured substrings -as new, separate, zero-terminated strings. These functions identify substrings -by number. The next section describes functions for extracting named -substrings. -</P> -<P> -A substring that contains a binary zero is correctly extracted and has a -further zero added on the end, but the result is not, of course, a C string. -However, you can process such a string by referring to the length that is -returned by <b>pcre_copy_substring()</b> and <b>pcre_get_substring()</b>. -Unfortunately, the interface to <b>pcre_get_substring_list()</b> is not adequate -for handling strings containing binary zeros, because the end of the final -string is not independently indicated. -</P> -<P> -The first three arguments are the same for all three of these functions: -<i>subject</i> is the subject string that has just been successfully matched, -<i>ovector</i> is a pointer to the vector of integer offsets that was passed to -<b>pcre_exec()</b>, and <i>stringcount</i> is the number of substrings that were -captured by the match, including the substring that matched the entire regular -expression. This is the value returned by <b>pcre_exec()</b> if it is greater -than zero. If <b>pcre_exec()</b> returned zero, indicating that it ran out of -space in <i>ovector</i>, the value passed as <i>stringcount</i> should be the -number of elements in the vector divided by three. -</P> -<P> -The functions <b>pcre_copy_substring()</b> and <b>pcre_get_substring()</b> -extract a single substring, whose number is given as <i>stringnumber</i>. A -value of zero extracts the substring that matched the entire pattern, whereas -higher values extract the captured substrings. For <b>pcre_copy_substring()</b>, -the string is placed in <i>buffer</i>, whose length is given by -<i>buffersize</i>, while for <b>pcre_get_substring()</b> a new block of memory is -obtained via <b>pcre_malloc</b>, and its address is returned via -<i>stringptr</i>. The yield of the function is the length of the string, not -including the terminating zero, or one of these error codes: -<pre> - PCRE_ERROR_NOMEMORY (-6) -</pre> -The buffer was too small for <b>pcre_copy_substring()</b>, or the attempt to get -memory failed for <b>pcre_get_substring()</b>. -<pre> - PCRE_ERROR_NOSUBSTRING (-7) -</pre> -There is no substring whose number is <i>stringnumber</i>. -</P> -<P> -The <b>pcre_get_substring_list()</b> function extracts all available substrings -and builds a list of pointers to them. All this is done in a single block of -memory that is obtained via <b>pcre_malloc</b>. The address of the memory block -is returned via <i>listptr</i>, which is also the start of the list of string -pointers. The end of the list is marked by a NULL pointer. The yield of the -function is zero if all went well, or the error code -<pre> - PCRE_ERROR_NOMEMORY (-6) -</pre> -if the attempt to get the memory block failed. -</P> -<P> -When any of these functions encounter a substring that is unset, which can -happen when capturing subpattern number <i>n+1</i> matches some part of the -subject, but subpattern <i>n</i> has not been used at all, they return an empty -string. This can be distinguished from a genuine zero-length substring by -inspecting the appropriate offset in <i>ovector</i>, which is negative for unset -substrings. -</P> -<P> -The two convenience functions <b>pcre_free_substring()</b> and -<b>pcre_free_substring_list()</b> can be used to free the memory returned by -a previous call of <b>pcre_get_substring()</b> or -<b>pcre_get_substring_list()</b>, respectively. They do nothing more than call -the function pointed to by <b>pcre_free</b>, which of course could be called -directly from a C program. However, PCRE is used in some situations where it is -linked via a special interface to another programming language that cannot use -<b>pcre_free</b> directly; it is for these cases that the functions are -provided. -</P> -<br><a name="SEC19" href="#TOC1">EXTRACTING CAPTURED SUBSTRINGS BY NAME</a><br> -<P> -<b>int pcre_get_stringnumber(const pcre *<i>code</i>,</b> -<b> const char *<i>name</i>);</b> -<br> -<br> -<b>int pcre_copy_named_substring(const pcre *<i>code</i>,</b> -<b> const char *<i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, const char *<i>stringname</i>,</b> -<b> char *<i>buffer</i>, int <i>buffersize</i>);</b> -<br> -<br> -<b>int pcre_get_named_substring(const pcre *<i>code</i>,</b> -<b> const char *<i>subject</i>, int *<i>ovector</i>,</b> -<b> int <i>stringcount</i>, const char *<i>stringname</i>,</b> -<b> const char **<i>stringptr</i>);</b> -</P> -<P> -To extract a substring by name, you first have to find associated number. -For example, for this pattern -<pre> - (a+)b(?<xxx>\d+)... -</pre> -the number of the subpattern called "xxx" is 2. If the name is known to be -unique (PCRE_DUPNAMES was not set), you can find the number from the name by -calling <b>pcre_get_stringnumber()</b>. The first argument is the compiled -pattern, and the second is the name. The yield of the function is the -subpattern number, or PCRE_ERROR_NOSUBSTRING (-7) if there is no subpattern of -that name. -</P> -<P> -Given the number, you can extract the substring directly, or use one of the -functions described in the previous section. For convenience, there are also -two functions that do the whole job. -</P> -<P> -Most of the arguments of <b>pcre_copy_named_substring()</b> and -<b>pcre_get_named_substring()</b> are the same as those for the similarly named -functions that extract by number. As these are described in the previous -section, they are not re-described here. There are just two differences: -</P> -<P> -First, instead of a substring number, a substring name is given. Second, there -is an extra argument, given at the start, which is a pointer to the compiled -pattern. This is needed in order to gain access to the name-to-number -translation table. -</P> -<P> -These functions call <b>pcre_get_stringnumber()</b>, and if it succeeds, they -then call <b>pcre_copy_substring()</b> or <b>pcre_get_substring()</b>, as -appropriate. <b>NOTE:</b> If PCRE_DUPNAMES is set and there are duplicate names, -the behaviour may not be what you want (see the next section). -</P> -<P> -<b>Warning:</b> If the pattern uses the (?| feature to set up multiple -subpatterns with the same number, as described in the -<a href="pcrepattern.html#dupsubpatternnumber">section on duplicate subpattern numbers</a> -in the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -page, you cannot use names to distinguish the different subpatterns, because -names are not included in the compiled code. The matching process uses only -numbers. For this reason, the use of different names for subpatterns of the -same number causes an error at compile time. -</P> -<br><a name="SEC20" href="#TOC1">DUPLICATE SUBPATTERN NAMES</a><br> -<P> -<b>int pcre_get_stringtable_entries(const pcre *<i>code</i>,</b> -<b> const char *<i>name</i>, char **<i>first</i>, char **<i>last</i>);</b> -</P> -<P> -When a pattern is compiled with the PCRE_DUPNAMES option, names for subpatterns -are not required to be unique. (Duplicate names are always allowed for -subpatterns with the same number, created by using the (?| feature. Indeed, if -such subpatterns are named, they are required to use the same names.) -</P> -<P> -Normally, patterns with duplicate names are such that in any one match, only -one of the named subpatterns participates. An example is shown in the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -documentation. -</P> -<P> -When duplicates are present, <b>pcre_copy_named_substring()</b> and -<b>pcre_get_named_substring()</b> return the first substring corresponding to -the given name that is set. If none are set, PCRE_ERROR_NOSUBSTRING (-7) is -returned; no data is returned. The <b>pcre_get_stringnumber()</b> function -returns one of the numbers that are associated with the name, but it is not -defined which it is. -</P> -<P> -If you want to get full details of all captured substrings for a given name, -you must use the <b>pcre_get_stringtable_entries()</b> function. The first -argument is the compiled pattern, and the second is the name. The third and -fourth are pointers to variables which are updated by the function. After it -has run, they point to the first and last entries in the name-to-number table -for the given name. The function itself returns the length of each entry, or -PCRE_ERROR_NOSUBSTRING (-7) if there are none. The format of the table is -described above in the section entitled <i>Information about a pattern</i> -<a href="#infoaboutpattern">above.</a> -Given all the relevant entries for the name, you can extract each of their -numbers, and hence the captured data, if any. -</P> -<br><a name="SEC21" href="#TOC1">FINDING ALL POSSIBLE MATCHES</a><br> -<P> -The traditional matching function uses a similar algorithm to Perl, which stops -when it finds the first match, starting at a given point in the subject. If you -want to find all possible matches, or the longest possible match, consider -using the alternative matching function (see below) instead. If you cannot use -the alternative function, but still need to find all possible matches, you -can kludge it up by making use of the callout facility, which is described in -the -<a href="pcrecallout.html"><b>pcrecallout</b></a> -documentation. -</P> -<P> -What you have to do is to insert a callout right at the end of the pattern. -When your callout function is called, extract and save the current matched -substring. Then return 1, which forces <b>pcre_exec()</b> to backtrack and try -other alternatives. Ultimately, when it runs out of matches, <b>pcre_exec()</b> -will yield PCRE_ERROR_NOMATCH. -</P> -<br><a name="SEC22" href="#TOC1">OBTAINING AN ESTIMATE OF STACK USAGE</a><br> -<P> -Matching certain patterns using <b>pcre_exec()</b> can use a lot of process -stack, which in certain environments can be rather limited in size. Some users -find it helpful to have an estimate of the amount of stack that is used by -<b>pcre_exec()</b>, to help them set recursion limits, as described in the -<a href="pcrestack.html"><b>pcrestack</b></a> -documentation. The estimate that is output by <b>pcretest</b> when called with -the <b>-m</b> and <b>-C</b> options is obtained by calling <b>pcre_exec</b> with -the values NULL, NULL, NULL, -999, and -999 for its first five arguments. -</P> -<P> -Normally, if its first argument is NULL, <b>pcre_exec()</b> immediately returns -the negative error code PCRE_ERROR_NULL, but with this special combination of -arguments, it returns instead a negative number whose absolute value is the -approximate stack frame size in bytes. (A negative number is used so that it is -clear that no match has happened.) The value is approximate because in some -cases, recursive calls to <b>pcre_exec()</b> occur when there are one or two -additional variables on the stack. -</P> -<P> -If PCRE has been compiled to use the heap instead of the stack for recursion, -the value returned is the size of each block that is obtained from the heap. -<a name="dfamatch"></a></P> -<br><a name="SEC23" href="#TOC1">MATCHING A PATTERN: THE ALTERNATIVE FUNCTION</a><br> -<P> -<b>int pcre_dfa_exec(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> -<b> const char *<i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> -<b> int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> -<b> int *<i>workspace</i>, int <i>wscount</i>);</b> -</P> -<P> -The function <b>pcre_dfa_exec()</b> is called to match a subject string against -a compiled pattern, using a matching algorithm that scans the subject string -just once, and does not backtrack. This has different characteristics to the -normal algorithm, and is not compatible with Perl. Some of the features of PCRE -patterns are not supported. Nevertheless, there are times when this kind of -matching can be useful. For a discussion of the two matching algorithms, and a -list of features that <b>pcre_dfa_exec()</b> does not support, see the -<a href="pcrematching.html"><b>pcrematching</b></a> -documentation. -</P> -<P> -The arguments for the <b>pcre_dfa_exec()</b> function are the same as for -<b>pcre_exec()</b>, plus two extras. The <i>ovector</i> argument is used in a -different way, and this is described below. The other common arguments are used -in the same way as for <b>pcre_exec()</b>, so their description is not repeated -here. -</P> -<P> -The two additional arguments provide workspace for the function. The workspace -vector should contain at least 20 elements. It is used for keeping track of -multiple paths through the pattern tree. More workspace will be needed for -patterns and subjects where there are a lot of potential matches. -</P> -<P> -Here is an example of a simple call to <b>pcre_dfa_exec()</b>: -<pre> - int rc; - int ovector[10]; - int wspace[20]; - rc = pcre_dfa_exec( - re, /* result of pcre_compile() */ - NULL, /* we didn't study the pattern */ - "some string", /* the subject string */ - 11, /* the length of the subject string */ - 0, /* start at offset 0 in the subject */ - 0, /* default options */ - ovector, /* vector of integers for substring information */ - 10, /* number of elements (NOT size in bytes) */ - wspace, /* working space vector */ - 20); /* number of elements (NOT size in bytes) */ -</PRE> -</P> -<br><b> -Option bits for <b>pcre_dfa_exec()</b> -</b><br> -<P> -The unused bits of the <i>options</i> argument for <b>pcre_dfa_exec()</b> must be -zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_<i>xxx</i>, -PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, -PCRE_NO_UTF8_CHECK, PCRE_BSR_ANYCRLF, PCRE_BSR_UNICODE, PCRE_NO_START_OPTIMIZE, -PCRE_PARTIAL_HARD, PCRE_PARTIAL_SOFT, PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. -All but the last four of these are exactly the same as for <b>pcre_exec()</b>, -so their description is not repeated here. -<pre> - PCRE_PARTIAL_HARD - PCRE_PARTIAL_SOFT -</pre> -These have the same general effect as they do for <b>pcre_exec()</b>, but the -details are slightly different. When PCRE_PARTIAL_HARD is set for -<b>pcre_dfa_exec()</b>, it returns PCRE_ERROR_PARTIAL if the end of the subject -is reached and there is still at least one matching possibility that requires -additional characters. This happens even if some complete matches have also -been found. When PCRE_PARTIAL_SOFT is set, the return code PCRE_ERROR_NOMATCH -is converted into PCRE_ERROR_PARTIAL if the end of the subject is reached, -there have been no complete matches, but there is still at least one matching -possibility. The portion of the string that was inspected when the longest -partial match was found is set as the first matching string in both cases. -There is a more detailed discussion of partial and multi-segment matching, with -examples, in the -<a href="pcrepartial.html"><b>pcrepartial</b></a> -documentation. -<pre> - PCRE_DFA_SHORTEST -</pre> -Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to stop as -soon as it has found one match. Because of the way the alternative algorithm -works, this is necessarily the shortest possible match at the first possible -matching point in the subject string. -<pre> - PCRE_DFA_RESTART -</pre> -When <b>pcre_dfa_exec()</b> returns a partial match, it is possible to call it -again, with additional subject characters, and have it continue with the same -match. The PCRE_DFA_RESTART option requests this action; when it is set, the -<i>workspace</i> and <i>wscount</i> options must reference the same vector as -before because data about the match so far is left in them after a partial -match. There is more discussion of this facility in the -<a href="pcrepartial.html"><b>pcrepartial</b></a> -documentation. -</P> -<br><b> -Successful returns from <b>pcre_dfa_exec()</b> -</b><br> -<P> -When <b>pcre_dfa_exec()</b> succeeds, it may have matched more than one -substring in the subject. Note, however, that all the matches from one run of -the function start at the same point in the subject. The shorter matches are -all initial substrings of the longer matches. For example, if the pattern -<pre> - <.*> -</pre> -is matched against the string -<pre> - This is <something> <something else> <something further> no more -</pre> -the three matched strings are -<pre> - <something> - <something> <something else> - <something> <something else> <something further> -</pre> -On success, the yield of the function is a number greater than zero, which is -the number of matched substrings. The substrings themselves are returned in -<i>ovector</i>. Each string uses two elements; the first is the offset to the -start, and the second is the offset to the end. In fact, all the strings have -the same start offset. (Space could have been saved by giving this only once, -but it was decided to retain some compatibility with the way <b>pcre_exec()</b> -returns data, even though the meaning of the strings is different.) -</P> -<P> -The strings are returned in reverse order of length; that is, the longest -matching string is given first. If there were too many matches to fit into -<i>ovector</i>, the yield of the function is zero, and the vector is filled with -the longest matches. Unlike <b>pcre_exec()</b>, <b>pcre_dfa_exec()</b> can use -the entire <i>ovector</i> for returning matched strings. -</P> -<P> -NOTE: PCRE's "auto-possessification" optimization usually applies to character -repeats at the end of a pattern (as well as internally). For example, the -pattern "a\d+" is compiled as if it were "a\d++" because there is no point -even considering the possibility of backtracking into the repeated digits. For -DFA matching, this means that only one possible match is found. If you really -do want multiple matches in such cases, either use an ungreedy repeat -("a\d+?") or set the PCRE_NO_AUTO_POSSESS option when compiling. -</P> -<br><b> -Error returns from <b>pcre_dfa_exec()</b> -</b><br> -<P> -The <b>pcre_dfa_exec()</b> function returns a negative number when it fails. -Many of the errors are the same as for <b>pcre_exec()</b>, and these are -described -<a href="#errorlist">above.</a> -There are in addition the following errors that are specific to -<b>pcre_dfa_exec()</b>: -<pre> - PCRE_ERROR_DFA_UITEM (-16) -</pre> -This return is given if <b>pcre_dfa_exec()</b> encounters an item in the pattern -that it does not support, for instance, the use of \C or a back reference. -<pre> - PCRE_ERROR_DFA_UCOND (-17) -</pre> -This return is given if <b>pcre_dfa_exec()</b> encounters a condition item that -uses a back reference for the condition, or a test for recursion in a specific -group. These are not supported. -<pre> - PCRE_ERROR_DFA_UMLIMIT (-18) -</pre> -This return is given if <b>pcre_dfa_exec()</b> is called with an <i>extra</i> -block that contains a setting of the <i>match_limit</i> or -<i>match_limit_recursion</i> fields. This is not supported (these fields are -meaningless for DFA matching). -<pre> - PCRE_ERROR_DFA_WSSIZE (-19) -</pre> -This return is given if <b>pcre_dfa_exec()</b> runs out of space in the -<i>workspace</i> vector. -<pre> - PCRE_ERROR_DFA_RECURSE (-20) -</pre> -When a recursive subpattern is processed, the matching function calls itself -recursively, using private vectors for <i>ovector</i> and <i>workspace</i>. This -error is given if the output vector is not large enough. This should be -extremely rare, as a vector of size 1000 is used. -<pre> - PCRE_ERROR_DFA_BADRESTART (-30) -</pre> -When <b>pcre_dfa_exec()</b> is called with the <b>PCRE_DFA_RESTART</b> option, -some plausibility checks are made on the contents of the workspace, which -should contain data about the previous partial match. If any of these checks -fail, this error is given. -</P> -<br><a name="SEC24" href="#TOC1">SEE ALSO</a><br> -<P> -<b>pcre16</b>(3), <b>pcre32</b>(3), <b>pcrebuild</b>(3), <b>pcrecallout</b>(3), -<b>pcrecpp(3)</b>(3), <b>pcrematching</b>(3), <b>pcrepartial</b>(3), -<b>pcreposix</b>(3), <b>pcreprecompile</b>(3), <b>pcresample</b>(3), -<b>pcrestack</b>(3). -</P> -<br><a name="SEC25" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC26" href="#TOC1">REVISION</a><br> -<P> -Last updated: 09 February 2014 -<br> -Copyright © 1997-2014 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcrebuild.html b/plugins/Pcre16/docs/doc/html/pcrebuild.html deleted file mode 100644 index 03c8cbe0b2..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcrebuild.html +++ /dev/null @@ -1,534 +0,0 @@ -<html> -<head> -<title>pcrebuild specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcrebuild man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">BUILDING PCRE</a> -<li><a name="TOC2" href="#SEC2">PCRE BUILD-TIME OPTIONS</a> -<li><a name="TOC3" href="#SEC3">BUILDING 8-BIT, 16-BIT AND 32-BIT LIBRARIES</a> -<li><a name="TOC4" href="#SEC4">BUILDING SHARED AND STATIC LIBRARIES</a> -<li><a name="TOC5" href="#SEC5">C++ SUPPORT</a> -<li><a name="TOC6" href="#SEC6">UTF-8, UTF-16 AND UTF-32 SUPPORT</a> -<li><a name="TOC7" href="#SEC7">UNICODE CHARACTER PROPERTY SUPPORT</a> -<li><a name="TOC8" href="#SEC8">JUST-IN-TIME COMPILER SUPPORT</a> -<li><a name="TOC9" href="#SEC9">CODE VALUE OF NEWLINE</a> -<li><a name="TOC10" href="#SEC10">WHAT \R MATCHES</a> -<li><a name="TOC11" href="#SEC11">POSIX MALLOC USAGE</a> -<li><a name="TOC12" href="#SEC12">HANDLING VERY LARGE PATTERNS</a> -<li><a name="TOC13" href="#SEC13">AVOIDING EXCESSIVE STACK USAGE</a> -<li><a name="TOC14" href="#SEC14">LIMITING PCRE RESOURCE USAGE</a> -<li><a name="TOC15" href="#SEC15">CREATING CHARACTER TABLES AT BUILD TIME</a> -<li><a name="TOC16" href="#SEC16">USING EBCDIC CODE</a> -<li><a name="TOC17" href="#SEC17">PCREGREP OPTIONS FOR COMPRESSED FILE SUPPORT</a> -<li><a name="TOC18" href="#SEC18">PCREGREP BUFFER SIZE</a> -<li><a name="TOC19" href="#SEC19">PCRETEST OPTION FOR LIBREADLINE SUPPORT</a> -<li><a name="TOC20" href="#SEC20">DEBUGGING WITH VALGRIND SUPPORT</a> -<li><a name="TOC21" href="#SEC21">CODE COVERAGE REPORTING</a> -<li><a name="TOC22" href="#SEC22">SEE ALSO</a> -<li><a name="TOC23" href="#SEC23">AUTHOR</a> -<li><a name="TOC24" href="#SEC24">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">BUILDING PCRE</a><br> -<P> -PCRE is distributed with a <b>configure</b> script that can be used to build the -library in Unix-like environments using the applications known as Autotools. -Also in the distribution are files to support building using <b>CMake</b> -instead of <b>configure</b>. The text file -<a href="README.txt"><b>README</b></a> -contains general information about building with Autotools (some of which is -repeated below), and also has some comments about building on various operating -systems. There is a lot more information about building PCRE without using -Autotools (including information about using <b>CMake</b> and building "by -hand") in the text file called -<a href="NON-AUTOTOOLS-BUILD.txt"><b>NON-AUTOTOOLS-BUILD</b>.</a> -You should consult this file as well as the -<a href="README.txt"><b>README</b></a> -file if you are building in a non-Unix-like environment. -</P> -<br><a name="SEC2" href="#TOC1">PCRE BUILD-TIME OPTIONS</a><br> -<P> -The rest of this document describes the optional features of PCRE that can be -selected when the library is compiled. It assumes use of the <b>configure</b> -script, where the optional features are selected or deselected by providing -options to <b>configure</b> before running the <b>make</b> command. However, the -same options can be selected in both Unix-like and non-Unix-like environments -using the GUI facility of <b>cmake-gui</b> if you are using <b>CMake</b> instead -of <b>configure</b> to build PCRE. -</P> -<P> -If you are not using Autotools or <b>CMake</b>, option selection can be done by -editing the <b>config.h</b> file, or by passing parameter settings to the -compiler, as described in -<a href="NON-AUTOTOOLS-BUILD.txt"><b>NON-AUTOTOOLS-BUILD</b>.</a> -</P> -<P> -The complete list of options for <b>configure</b> (which includes the standard -ones such as the selection of the installation directory) can be obtained by -running -<pre> - ./configure --help -</pre> -The following sections include descriptions of options whose names begin with ---enable or --disable. These settings specify changes to the defaults for the -<b>configure</b> command. Because of the way that <b>configure</b> works, ---enable and --disable always come in pairs, so the complementary option always -exists as well, but as it specifies the default, it is not described. -</P> -<br><a name="SEC3" href="#TOC1">BUILDING 8-BIT, 16-BIT AND 32-BIT LIBRARIES</a><br> -<P> -By default, a library called <b>libpcre</b> is built, containing functions that -take string arguments contained in vectors of bytes, either as single-byte -characters, or interpreted as UTF-8 strings. You can also build a separate -library, called <b>libpcre16</b>, in which strings are contained in vectors of -16-bit data units and interpreted either as single-unit characters or UTF-16 -strings, by adding -<pre> - --enable-pcre16 -</pre> -to the <b>configure</b> command. You can also build yet another separate -library, called <b>libpcre32</b>, in which strings are contained in vectors of -32-bit data units and interpreted either as single-unit characters or UTF-32 -strings, by adding -<pre> - --enable-pcre32 -</pre> -to the <b>configure</b> command. If you do not want the 8-bit library, add -<pre> - --disable-pcre8 -</pre> -as well. At least one of the three libraries must be built. Note that the C++ -and POSIX wrappers are for the 8-bit library only, and that <b>pcregrep</b> is -an 8-bit program. None of these are built if you select only the 16-bit or -32-bit libraries. -</P> -<br><a name="SEC4" href="#TOC1">BUILDING SHARED AND STATIC LIBRARIES</a><br> -<P> -The Autotools PCRE building process uses <b>libtool</b> to build both shared and -static libraries by default. You can suppress one of these by adding one of -<pre> - --disable-shared - --disable-static -</pre> -to the <b>configure</b> command, as required. -</P> -<br><a name="SEC5" href="#TOC1">C++ SUPPORT</a><br> -<P> -By default, if the 8-bit library is being built, the <b>configure</b> script -will search for a C++ compiler and C++ header files. If it finds them, it -automatically builds the C++ wrapper library (which supports only 8-bit -strings). You can disable this by adding -<pre> - --disable-cpp -</pre> -to the <b>configure</b> command. -</P> -<br><a name="SEC6" href="#TOC1">UTF-8, UTF-16 AND UTF-32 SUPPORT</a><br> -<P> -To build PCRE with support for UTF Unicode character strings, add -<pre> - --enable-utf -</pre> -to the <b>configure</b> command. This setting applies to all three libraries, -adding support for UTF-8 to the 8-bit library, support for UTF-16 to the 16-bit -library, and support for UTF-32 to the to the 32-bit library. There are no -separate options for enabling UTF-8, UTF-16 and UTF-32 independently because -that would allow ridiculous settings such as requesting UTF-16 support while -building only the 8-bit library. It is not possible to build one library with -UTF support and another without in the same configuration. (For backwards -compatibility, --enable-utf8 is a synonym of --enable-utf.) -</P> -<P> -Of itself, this setting does not make PCRE treat strings as UTF-8, UTF-16 or -UTF-32. As well as compiling PCRE with this option, you also have have to set -the PCRE_UTF8, PCRE_UTF16 or PCRE_UTF32 option (as appropriate) when you call -one of the pattern compiling functions. -</P> -<P> -If you set --enable-utf when compiling in an EBCDIC environment, PCRE expects -its input to be either ASCII or UTF-8 (depending on the run-time option). It is -not possible to support both EBCDIC and UTF-8 codes in the same version of the -library. Consequently, --enable-utf and --enable-ebcdic are mutually -exclusive. -</P> -<br><a name="SEC7" href="#TOC1">UNICODE CHARACTER PROPERTY SUPPORT</a><br> -<P> -UTF support allows the libraries to process character codepoints up to 0x10ffff -in the strings that they handle. On its own, however, it does not provide any -facilities for accessing the properties of such characters. If you want to be -able to use the pattern escapes \P, \p, and \X, which refer to Unicode -character properties, you must add -<pre> - --enable-unicode-properties -</pre> -to the <b>configure</b> command. This implies UTF support, even if you have -not explicitly requested it. -</P> -<P> -Including Unicode property support adds around 30K of tables to the PCRE -library. Only the general category properties such as <i>Lu</i> and <i>Nd</i> are -supported. Details are given in the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -documentation. -</P> -<br><a name="SEC8" href="#TOC1">JUST-IN-TIME COMPILER SUPPORT</a><br> -<P> -Just-in-time compiler support is included in the build by specifying -<pre> - --enable-jit -</pre> -This support is available only for certain hardware architectures. If this -option is set for an unsupported architecture, a compile time error occurs. -See the -<a href="pcrejit.html"><b>pcrejit</b></a> -documentation for a discussion of JIT usage. When JIT support is enabled, -pcregrep automatically makes use of it, unless you add -<pre> - --disable-pcregrep-jit -</pre> -to the "configure" command. -</P> -<br><a name="SEC9" href="#TOC1">CODE VALUE OF NEWLINE</a><br> -<P> -By default, PCRE interprets the linefeed (LF) character as indicating the end -of a line. This is the normal newline character on Unix-like systems. You can -compile PCRE to use carriage return (CR) instead, by adding -<pre> - --enable-newline-is-cr -</pre> -to the <b>configure</b> command. There is also a --enable-newline-is-lf option, -which explicitly specifies linefeed as the newline character. -<br> -<br> -Alternatively, you can specify that line endings are to be indicated by the two -character sequence CRLF. If you want this, add -<pre> - --enable-newline-is-crlf -</pre> -to the <b>configure</b> command. There is a fourth option, specified by -<pre> - --enable-newline-is-anycrlf -</pre> -which causes PCRE to recognize any of the three sequences CR, LF, or CRLF as -indicating a line ending. Finally, a fifth option, specified by -<pre> - --enable-newline-is-any -</pre> -causes PCRE to recognize any Unicode newline sequence. -</P> -<P> -Whatever line ending convention is selected when PCRE is built can be -overridden when the library functions are called. At build time it is -conventional to use the standard for your operating system. -</P> -<br><a name="SEC10" href="#TOC1">WHAT \R MATCHES</a><br> -<P> -By default, the sequence \R in a pattern matches any Unicode newline sequence, -whatever has been selected as the line ending sequence. If you specify -<pre> - --enable-bsr-anycrlf -</pre> -the default is changed so that \R matches only CR, LF, or CRLF. Whatever is -selected when PCRE is built can be overridden when the library functions are -called. -</P> -<br><a name="SEC11" href="#TOC1">POSIX MALLOC USAGE</a><br> -<P> -When the 8-bit library is called through the POSIX interface (see the -<a href="pcreposix.html"><b>pcreposix</b></a> -documentation), additional working storage is required for holding the pointers -to capturing substrings, because PCRE requires three integers per substring, -whereas the POSIX interface provides only two. If the number of expected -substrings is small, the wrapper function uses space on the stack, because this -is faster than using <b>malloc()</b> for each call. The default threshold above -which the stack is no longer used is 10; it can be changed by adding a setting -such as -<pre> - --with-posix-malloc-threshold=20 -</pre> -to the <b>configure</b> command. -</P> -<br><a name="SEC12" href="#TOC1">HANDLING VERY LARGE PATTERNS</a><br> -<P> -Within a compiled pattern, offset values are used to point from one part to -another (for example, from an opening parenthesis to an alternation -metacharacter). By default, in the 8-bit and 16-bit libraries, two-byte values -are used for these offsets, leading to a maximum size for a compiled pattern of -around 64K. This is sufficient to handle all but the most gigantic patterns. -Nevertheless, some people do want to process truly enormous patterns, so it is -possible to compile PCRE to use three-byte or four-byte offsets by adding a -setting such as -<pre> - --with-link-size=3 -</pre> -to the <b>configure</b> command. The value given must be 2, 3, or 4. For the -16-bit library, a value of 3 is rounded up to 4. In these libraries, using -longer offsets slows down the operation of PCRE because it has to load -additional data when handling them. For the 32-bit library the value is always -4 and cannot be overridden; the value of --with-link-size is ignored. -</P> -<br><a name="SEC13" href="#TOC1">AVOIDING EXCESSIVE STACK USAGE</a><br> -<P> -When matching with the <b>pcre_exec()</b> function, PCRE implements backtracking -by making recursive calls to an internal function called <b>match()</b>. In -environments where the size of the stack is limited, this can severely limit -PCRE's operation. (The Unix environment does not usually suffer from this -problem, but it may sometimes be necessary to increase the maximum stack size. -There is a discussion in the -<a href="pcrestack.html"><b>pcrestack</b></a> -documentation.) An alternative approach to recursion that uses memory from the -heap to remember data, instead of using recursive function calls, has been -implemented to work round the problem of limited stack size. If you want to -build a version of PCRE that works this way, add -<pre> - --disable-stack-for-recursion -</pre> -to the <b>configure</b> command. With this configuration, PCRE will use the -<b>pcre_stack_malloc</b> and <b>pcre_stack_free</b> variables to call memory -management functions. By default these point to <b>malloc()</b> and -<b>free()</b>, but you can replace the pointers so that your own functions are -used instead. -</P> -<P> -Separate functions are provided rather than using <b>pcre_malloc</b> and -<b>pcre_free</b> because the usage is very predictable: the block sizes -requested are always the same, and the blocks are always freed in reverse -order. A calling program might be able to implement optimized functions that -perform better than <b>malloc()</b> and <b>free()</b>. PCRE runs noticeably more -slowly when built in this way. This option affects only the <b>pcre_exec()</b> -function; it is not relevant for <b>pcre_dfa_exec()</b>. -</P> -<br><a name="SEC14" href="#TOC1">LIMITING PCRE RESOURCE USAGE</a><br> -<P> -Internally, PCRE has a function called <b>match()</b>, which it calls repeatedly -(sometimes recursively) when matching a pattern with the <b>pcre_exec()</b> -function. By controlling the maximum number of times this function may be -called during a single matching operation, a limit can be placed on the -resources used by a single call to <b>pcre_exec()</b>. The limit can be changed -at run time, as described in the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation. The default is 10 million, but this can be changed by adding a -setting such as -<pre> - --with-match-limit=500000 -</pre> -to the <b>configure</b> command. This setting has no effect on the -<b>pcre_dfa_exec()</b> matching function. -</P> -<P> -In some environments it is desirable to limit the depth of recursive calls of -<b>match()</b> more strictly than the total number of calls, in order to -restrict the maximum amount of stack (or heap, if --disable-stack-for-recursion -is specified) that is used. A second limit controls this; it defaults to the -value that is set for --with-match-limit, which imposes no additional -constraints. However, you can set a lower limit by adding, for example, -<pre> - --with-match-limit-recursion=10000 -</pre> -to the <b>configure</b> command. This value can also be overridden at run time. -</P> -<br><a name="SEC15" href="#TOC1">CREATING CHARACTER TABLES AT BUILD TIME</a><br> -<P> -PCRE uses fixed tables for processing characters whose code values are less -than 256. By default, PCRE is built with a set of tables that are distributed -in the file <i>pcre_chartables.c.dist</i>. These tables are for ASCII codes -only. If you add -<pre> - --enable-rebuild-chartables -</pre> -to the <b>configure</b> command, the distributed tables are no longer used. -Instead, a program called <b>dftables</b> is compiled and run. This outputs the -source for new set of tables, created in the default locale of your C run-time -system. (This method of replacing the tables does not work if you are cross -compiling, because <b>dftables</b> is run on the local host. If you need to -create alternative tables when cross compiling, you will have to do so "by -hand".) -</P> -<br><a name="SEC16" href="#TOC1">USING EBCDIC CODE</a><br> -<P> -PCRE assumes by default that it will run in an environment where the character -code is ASCII (or Unicode, which is a superset of ASCII). This is the case for -most computer operating systems. PCRE can, however, be compiled to run in an -EBCDIC environment by adding -<pre> - --enable-ebcdic -</pre> -to the <b>configure</b> command. This setting implies ---enable-rebuild-chartables. You should only use it if you know that you are in -an EBCDIC environment (for example, an IBM mainframe operating system). The ---enable-ebcdic option is incompatible with --enable-utf. -</P> -<P> -The EBCDIC character that corresponds to an ASCII LF is assumed to have the -value 0x15 by default. However, in some EBCDIC environments, 0x25 is used. In -such an environment you should use -<pre> - --enable-ebcdic-nl25 -</pre> -as well as, or instead of, --enable-ebcdic. The EBCDIC character for CR has the -same value as in ASCII, namely, 0x0d. Whichever of 0x15 and 0x25 is <i>not</i> -chosen as LF is made to correspond to the Unicode NEL character (which, in -Unicode, is 0x85). -</P> -<P> -The options that select newline behaviour, such as --enable-newline-is-cr, -and equivalent run-time options, refer to these character values in an EBCDIC -environment. -</P> -<br><a name="SEC17" href="#TOC1">PCREGREP OPTIONS FOR COMPRESSED FILE SUPPORT</a><br> -<P> -By default, <b>pcregrep</b> reads all files as plain text. You can build it so -that it recognizes files whose names end in <b>.gz</b> or <b>.bz2</b>, and reads -them with <b>libz</b> or <b>libbz2</b>, respectively, by adding one or both of -<pre> - --enable-pcregrep-libz - --enable-pcregrep-libbz2 -</pre> -to the <b>configure</b> command. These options naturally require that the -relevant libraries are installed on your system. Configuration will fail if -they are not. -</P> -<br><a name="SEC18" href="#TOC1">PCREGREP BUFFER SIZE</a><br> -<P> -<b>pcregrep</b> uses an internal buffer to hold a "window" on the file it is -scanning, in order to be able to output "before" and "after" lines when it -finds a match. The size of the buffer is controlled by a parameter whose -default value is 20K. The buffer itself is three times this size, but because -of the way it is used for holding "before" lines, the longest line that is -guaranteed to be processable is the parameter size. You can change the default -parameter value by adding, for example, -<pre> - --with-pcregrep-bufsize=50K -</pre> -to the <b>configure</b> command. The caller of \fPpcregrep\fP can, however, -override this value by specifying a run-time option. -</P> -<br><a name="SEC19" href="#TOC1">PCRETEST OPTION FOR LIBREADLINE SUPPORT</a><br> -<P> -If you add -<pre> - --enable-pcretest-libreadline -</pre> -to the <b>configure</b> command, <b>pcretest</b> is linked with the -<b>libreadline</b> library, and when its input is from a terminal, it reads it -using the <b>readline()</b> function. This provides line-editing and history -facilities. Note that <b>libreadline</b> is GPL-licensed, so if you distribute a -binary of <b>pcretest</b> linked in this way, there may be licensing issues. -</P> -<P> -Setting this option causes the <b>-lreadline</b> option to be added to the -<b>pcretest</b> build. In many operating environments with a sytem-installed -<b>libreadline</b> this is sufficient. However, in some environments (e.g. -if an unmodified distribution version of readline is in use), some extra -configuration may be necessary. The INSTALL file for <b>libreadline</b> says -this: -<pre> - "Readline uses the termcap functions, but does not link with the - termcap or curses library itself, allowing applications which link - with readline the to choose an appropriate library." -</pre> -If your environment has not been set up so that an appropriate library is -automatically included, you may need to add something like -<pre> - LIBS="-ncurses" -</pre> -immediately before the <b>configure</b> command. -</P> -<br><a name="SEC20" href="#TOC1">DEBUGGING WITH VALGRIND SUPPORT</a><br> -<P> -By adding the -<pre> - --enable-valgrind -</pre> -option to to the <b>configure</b> command, PCRE will use valgrind annotations -to mark certain memory regions as unaddressable. This allows it to detect -invalid memory accesses, and is mostly useful for debugging PCRE itself. -</P> -<br><a name="SEC21" href="#TOC1">CODE COVERAGE REPORTING</a><br> -<P> -If your C compiler is gcc, you can build a version of PCRE that can generate a -code coverage report for its test suite. To enable this, you must install -<b>lcov</b> version 1.6 or above. Then specify -<pre> - --enable-coverage -</pre> -to the <b>configure</b> command and build PCRE in the usual way. -</P> -<P> -Note that using <b>ccache</b> (a caching C compiler) is incompatible with code -coverage reporting. If you have configured <b>ccache</b> to run automatically -on your system, you must set the environment variable -<pre> - CCACHE_DISABLE=1 -</pre> -before running <b>make</b> to build PCRE, so that <b>ccache</b> is not used. -</P> -<P> -When --enable-coverage is used, the following addition targets are added to the -<i>Makefile</i>: -<pre> - make coverage -</pre> -This creates a fresh coverage report for the PCRE test suite. It is equivalent -to running "make coverage-reset", "make coverage-baseline", "make check", and -then "make coverage-report". -<pre> - make coverage-reset -</pre> -This zeroes the coverage counters, but does nothing else. -<pre> - make coverage-baseline -</pre> -This captures baseline coverage information. -<pre> - make coverage-report -</pre> -This creates the coverage report. -<pre> - make coverage-clean-report -</pre> -This removes the generated coverage report without cleaning the coverage data -itself. -<pre> - make coverage-clean-data -</pre> -This removes the captured coverage data without removing the coverage files -created at compile time (*.gcno). -<pre> - make coverage-clean -</pre> -This cleans all coverage data including the generated coverage report. For more -information about code coverage, see the <b>gcov</b> and <b>lcov</b> -documentation. -</P> -<br><a name="SEC22" href="#TOC1">SEE ALSO</a><br> -<P> -<b>pcreapi</b>(3), <b>pcre16</b>, <b>pcre32</b>, <b>pcre_config</b>(3). -</P> -<br><a name="SEC23" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC24" href="#TOC1">REVISION</a><br> -<P> -Last updated: 12 May 2013 -<br> -Copyright © 1997-2013 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcrecallout.html b/plugins/Pcre16/docs/doc/html/pcrecallout.html deleted file mode 100644 index 53a937f52d..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcrecallout.html +++ /dev/null @@ -1,286 +0,0 @@ -<html> -<head> -<title>pcrecallout specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcrecallout man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">SYNOPSIS</a> -<li><a name="TOC2" href="#SEC2">DESCRIPTION</a> -<li><a name="TOC3" href="#SEC3">MISSING CALLOUTS</a> -<li><a name="TOC4" href="#SEC4">THE CALLOUT INTERFACE</a> -<li><a name="TOC5" href="#SEC5">RETURN VALUES</a> -<li><a name="TOC6" href="#SEC6">AUTHOR</a> -<li><a name="TOC7" href="#SEC7">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">SYNOPSIS</a><br> -<P> -<b>#include <pcre.h></b> -</P> -<P> -<b>int (*pcre_callout)(pcre_callout_block *);</b> -</P> -<P> -<b>int (*pcre16_callout)(pcre16_callout_block *);</b> -</P> -<P> -<b>int (*pcre32_callout)(pcre32_callout_block *);</b> -</P> -<br><a name="SEC2" href="#TOC1">DESCRIPTION</a><br> -<P> -PCRE provides a feature called "callout", which is a means of temporarily -passing control to the caller of PCRE in the middle of pattern matching. The -caller of PCRE provides an external function by putting its entry point in the -global variable <i>pcre_callout</i> (<i>pcre16_callout</i> for the 16-bit -library, <i>pcre32_callout</i> for the 32-bit library). By default, this -variable contains NULL, which disables all calling out. -</P> -<P> -Within a regular expression, (?C) indicates the points at which the external -function is to be called. Different callout points can be identified by putting -a number less than 256 after the letter C. The default value is zero. -For example, this pattern has two callout points: -<pre> - (?C1)abc(?C2)def -</pre> -If the PCRE_AUTO_CALLOUT option bit is set when a pattern is compiled, PCRE -automatically inserts callouts, all with number 255, before each item in the -pattern. For example, if PCRE_AUTO_CALLOUT is used with the pattern -<pre> - A(\d{2}|--) -</pre> -it is processed as if it were -<br> -<br> -(?C255)A(?C255)((?C255)\d{2}(?C255)|(?C255)-(?C255)-(?C255))(?C255) -<br> -<br> -Notice that there is a callout before and after each parenthesis and -alternation bar. If the pattern contains a conditional group whose condition is -an assertion, an automatic callout is inserted immediately before the -condition. Such a callout may also be inserted explicitly, for example: -<pre> - (?(?C9)(?=a)ab|de) -</pre> -This applies only to assertion conditions (because they are themselves -independent groups). -</P> -<P> -Automatic callouts can be used for tracking the progress of pattern matching. -The -<a href="pcretest.html"><b>pcretest</b></a> -program has a pattern qualifier (/C) that sets automatic callouts; when it is -used, the output indicates how the pattern is being matched. This is useful -information when you are trying to optimize the performance of a particular -pattern. -</P> -<br><a name="SEC3" href="#TOC1">MISSING CALLOUTS</a><br> -<P> -You should be aware that, because of optimizations in the way PCRE compiles and -matches patterns, callouts sometimes do not happen exactly as you might expect. -</P> -<P> -At compile time, PCRE "auto-possessifies" repeated items when it knows that -what follows cannot be part of the repeat. For example, a+[bc] is compiled as -if it were a++[bc]. The <b>pcretest</b> output when this pattern is anchored and -then applied with automatic callouts to the string "aaaa" is: -<pre> - --->aaaa - +0 ^ ^ - +1 ^ a+ - +3 ^ ^ [bc] - No match -</pre> -This indicates that when matching [bc] fails, there is no backtracking into a+ -and therefore the callouts that would be taken for the backtracks do not occur. -You can disable the auto-possessify feature by passing PCRE_NO_AUTO_POSSESS -to <b>pcre_compile()</b>, or starting the pattern with (*NO_AUTO_POSSESS). If -this is done in <b>pcretest</b> (using the /O qualifier), the output changes to -this: -<pre> - --->aaaa - +0 ^ ^ - +1 ^ a+ - +3 ^ ^ [bc] - +3 ^ ^ [bc] - +3 ^ ^ [bc] - +3 ^^ [bc] - No match -</pre> -This time, when matching [bc] fails, the matcher backtracks into a+ and tries -again, repeatedly, until a+ itself fails. -</P> -<P> -Other optimizations that provide fast "no match" results also affect callouts. -For example, if the pattern is -<pre> - ab(?C4)cd -</pre> -PCRE knows that any matching string must contain the letter "d". If the subject -string is "abyz", the lack of "d" means that matching doesn't ever start, and -the callout is never reached. However, with "abyd", though the result is still -no match, the callout is obeyed. -</P> -<P> -If the pattern is studied, PCRE knows the minimum length of a matching string, -and will immediately give a "no match" return without actually running a match -if the subject is not long enough, or, for unanchored patterns, if it has -been scanned far enough. -</P> -<P> -You can disable these optimizations by passing the PCRE_NO_START_OPTIMIZE -option to the matching function, or by starting the pattern with -(*NO_START_OPT). This slows down the matching process, but does ensure that -callouts such as the example above are obeyed. -</P> -<br><a name="SEC4" href="#TOC1">THE CALLOUT INTERFACE</a><br> -<P> -During matching, when PCRE reaches a callout point, the external function -defined by <i>pcre_callout</i> or <i>pcre[16|32]_callout</i> is called (if it is -set). This applies to both normal and DFA matching. The only argument to the -callout function is a pointer to a <b>pcre_callout</b> or -<b>pcre[16|32]_callout</b> block. These structures contains the following -fields: -<pre> - int <i>version</i>; - int <i>callout_number</i>; - int *<i>offset_vector</i>; - const char *<i>subject</i>; (8-bit version) - PCRE_SPTR16 <i>subject</i>; (16-bit version) - PCRE_SPTR32 <i>subject</i>; (32-bit version) - int <i>subject_length</i>; - int <i>start_match</i>; - int <i>current_position</i>; - int <i>capture_top</i>; - int <i>capture_last</i>; - void *<i>callout_data</i>; - int <i>pattern_position</i>; - int <i>next_item_length</i>; - const unsigned char *<i>mark</i>; (8-bit version) - const PCRE_UCHAR16 *<i>mark</i>; (16-bit version) - const PCRE_UCHAR32 *<i>mark</i>; (32-bit version) -</pre> -The <i>version</i> field is an integer containing the version number of the -block format. The initial version was 0; the current version is 2. The version -number will change again in future if additional fields are added, but the -intention is never to remove any of the existing fields. -</P> -<P> -The <i>callout_number</i> field contains the number of the callout, as compiled -into the pattern (that is, the number after ?C for manual callouts, and 255 for -automatically generated callouts). -</P> -<P> -The <i>offset_vector</i> field is a pointer to the vector of offsets that was -passed by the caller to the matching function. When <b>pcre_exec()</b> or -<b>pcre[16|32]_exec()</b> is used, the contents can be inspected, in order to -extract substrings that have been matched so far, in the same way as for -extracting substrings after a match has completed. For the DFA matching -functions, this field is not useful. -</P> -<P> -The <i>subject</i> and <i>subject_length</i> fields contain copies of the values -that were passed to the matching function. -</P> -<P> -The <i>start_match</i> field normally contains the offset within the subject at -which the current match attempt started. However, if the escape sequence \K -has been encountered, this value is changed to reflect the modified starting -point. If the pattern is not anchored, the callout function may be called -several times from the same point in the pattern for different starting points -in the subject. -</P> -<P> -The <i>current_position</i> field contains the offset within the subject of the -current match pointer. -</P> -<P> -When the <b>pcre_exec()</b> or <b>pcre[16|32]_exec()</b> is used, the -<i>capture_top</i> field contains one more than the number of the highest -numbered captured substring so far. If no substrings have been captured, the -value of <i>capture_top</i> is one. This is always the case when the DFA -functions are used, because they do not support captured substrings. -</P> -<P> -The <i>capture_last</i> field contains the number of the most recently captured -substring. However, when a recursion exits, the value reverts to what it was -outside the recursion, as do the values of all captured substrings. If no -substrings have been captured, the value of <i>capture_last</i> is -1. This is -always the case for the DFA matching functions. -</P> -<P> -The <i>callout_data</i> field contains a value that is passed to a matching -function specifically so that it can be passed back in callouts. It is passed -in the <i>callout_data</i> field of a <b>pcre_extra</b> or <b>pcre[16|32]_extra</b> -data structure. If no such data was passed, the value of <i>callout_data</i> in -a callout block is NULL. There is a description of the <b>pcre_extra</b> -structure in the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation. -</P> -<P> -The <i>pattern_position</i> field is present from version 1 of the callout -structure. It contains the offset to the next item to be matched in the pattern -string. -</P> -<P> -The <i>next_item_length</i> field is present from version 1 of the callout -structure. It contains the length of the next item to be matched in the pattern -string. When the callout immediately precedes an alternation bar, a closing -parenthesis, or the end of the pattern, the length is zero. When the callout -precedes an opening parenthesis, the length is that of the entire subpattern. -</P> -<P> -The <i>pattern_position</i> and <i>next_item_length</i> fields are intended to -help in distinguishing between different automatic callouts, which all have the -same callout number. However, they are set for all callouts. -</P> -<P> -The <i>mark</i> field is present from version 2 of the callout structure. In -callouts from <b>pcre_exec()</b> or <b>pcre[16|32]_exec()</b> it contains a -pointer to the zero-terminated name of the most recently passed (*MARK), -(*PRUNE), or (*THEN) item in the match, or NULL if no such items have been -passed. Instances of (*PRUNE) or (*THEN) without a name do not obliterate a -previous (*MARK). In callouts from the DFA matching functions this field always -contains NULL. -</P> -<br><a name="SEC5" href="#TOC1">RETURN VALUES</a><br> -<P> -The external callout function returns an integer to PCRE. If the value is zero, -matching proceeds as normal. If the value is greater than zero, matching fails -at the current point, but the testing of other matching possibilities goes -ahead, just as if a lookahead assertion had failed. If the value is less than -zero, the match is abandoned, the matching function returns the negative value. -</P> -<P> -Negative values should normally be chosen from the set of PCRE_ERROR_xxx -values. In particular, PCRE_ERROR_NOMATCH forces a standard "no match" failure. -The error number PCRE_ERROR_CALLOUT is reserved for use by callout functions; -it will never be used by PCRE itself. -</P> -<br><a name="SEC6" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC7" href="#TOC1">REVISION</a><br> -<P> -Last updated: 12 November 2013 -<br> -Copyright © 1997-2013 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcrecompat.html b/plugins/Pcre16/docs/doc/html/pcrecompat.html deleted file mode 100644 index 3e6226692e..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcrecompat.html +++ /dev/null @@ -1,235 +0,0 @@ -<html> -<head> -<title>pcrecompat specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcrecompat man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -DIFFERENCES BETWEEN PCRE AND PERL -</b><br> -<P> -This document describes the differences in the ways that PCRE and Perl handle -regular expressions. The differences described here are with respect to Perl -versions 5.10 and above. -</P> -<P> -1. PCRE has only a subset of Perl's Unicode support. Details of what it does -have are given in the -<a href="pcreunicode.html"><b>pcreunicode</b></a> -page. -</P> -<P> -2. PCRE allows repeat quantifiers only on parenthesized assertions, but they do -not mean what you might think. For example, (?!a){3} does not assert that the -next three characters are not "a". It just asserts that the next character is -not "a" three times (in principle: PCRE optimizes this to run the assertion -just once). Perl allows repeat quantifiers on other assertions such as \b, but -these do not seem to have any use. -</P> -<P> -3. Capturing subpatterns that occur inside negative lookahead assertions are -counted, but their entries in the offsets vector are never set. Perl sometimes -(but not always) sets its numerical variables from inside negative assertions. -</P> -<P> -4. Though binary zero characters are supported in the subject string, they are -not allowed in a pattern string because it is passed as a normal C string, -terminated by zero. The escape sequence \0 can be used in the pattern to -represent a binary zero. -</P> -<P> -5. The following Perl escape sequences are not supported: \l, \u, \L, -\U, and \N when followed by a character name or Unicode value. (\N on its -own, matching a non-newline character, is supported.) In fact these are -implemented by Perl's general string-handling and are not part of its pattern -matching engine. If any of these are encountered by PCRE, an error is -generated by default. However, if the PCRE_JAVASCRIPT_COMPAT option is set, -\U and \u are interpreted as JavaScript interprets them. -</P> -<P> -6. The Perl escape sequences \p, \P, and \X are supported only if PCRE is -built with Unicode character property support. The properties that can be -tested with \p and \P are limited to the general category properties such as -Lu and Nd, script names such as Greek or Han, and the derived properties Any -and L&. PCRE does support the Cs (surrogate) property, which Perl does not; the -Perl documentation says "Because Perl hides the need for the user to understand -the internal representation of Unicode characters, there is no need to -implement the somewhat messy concept of surrogates." -</P> -<P> -7. PCRE does support the \Q...\E escape for quoting substrings. Characters in -between are treated as literals. This is slightly different from Perl in that $ -and @ are also handled as literals inside the quotes. In Perl, they cause -variable interpolation (but of course PCRE does not have variables). Note the -following examples: -<pre> - Pattern PCRE matches Perl matches - - \Qabc$xyz\E abc$xyz abc followed by the contents of $xyz - \Qabc\$xyz\E abc\$xyz abc\$xyz - \Qabc\E\$\Qxyz\E abc$xyz abc$xyz -</pre> -The \Q...\E sequence is recognized both inside and outside character classes. -</P> -<P> -8. Fairly obviously, PCRE does not support the (?{code}) and (??{code}) -constructions. However, there is support for recursive patterns. This is not -available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE "callout" -feature allows an external function to be called during pattern matching. See -the -<a href="pcrecallout.html"><b>pcrecallout</b></a> -documentation for details. -</P> -<P> -9. Subpatterns that are called as subroutines (whether or not recursively) are -always treated as atomic groups in PCRE. This is like Python, but unlike Perl. -Captured values that are set outside a subroutine call can be reference from -inside in PCRE, but not in Perl. There is a discussion that explains these -differences in more detail in the -<a href="pcrepattern.html#recursiondifference">section on recursion differences from Perl</a> -in the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -page. -</P> -<P> -10. If any of the backtracking control verbs are used in a subpattern that is -called as a subroutine (whether or not recursively), their effect is confined -to that subpattern; it does not extend to the surrounding pattern. This is not -always the case in Perl. In particular, if (*THEN) is present in a group that -is called as a subroutine, its action is limited to that group, even if the -group does not contain any | characters. Note that such subpatterns are -processed as anchored at the point where they are tested. -</P> -<P> -11. If a pattern contains more than one backtracking control verb, the first -one that is backtracked onto acts. For example, in the pattern -A(*COMMIT)B(*PRUNE)C a failure in B triggers (*COMMIT), but a failure in C -triggers (*PRUNE). Perl's behaviour is more complex; in many cases it is the -same as PCRE, but there are examples where it differs. -</P> -<P> -12. Most backtracking verbs in assertions have their normal actions. They are -not confined to the assertion. -</P> -<P> -13. There are some differences that are concerned with the settings of captured -strings when part of a pattern is repeated. For example, matching "aba" against -the pattern /^(a(b)?)+$/ in Perl leaves $2 unset, but in PCRE it is set to "b". -</P> -<P> -14. PCRE's handling of duplicate subpattern numbers and duplicate subpattern -names is not as general as Perl's. This is a consequence of the fact the PCRE -works internally just with numbers, using an external table to translate -between numbers and names. In particular, a pattern such as (?|(?<a>A)|(?<b)B), -where the two capturing parentheses have the same number but different names, -is not supported, and causes an error at compile time. If it were allowed, it -would not be possible to distinguish which parentheses matched, because both -names map to capturing subpattern number 1. To avoid this confusing situation, -an error is given at compile time. -</P> -<P> -15. Perl recognizes comments in some places that PCRE does not, for example, -between the ( and ? at the start of a subpattern. If the /x modifier is set, -Perl allows white space between ( and ? (though current Perls warn that this is -deprecated) but PCRE never does, even if the PCRE_EXTENDED option is set. -</P> -<P> -16. Perl, when in warning mode, gives warnings for character classes such as -[A-\d] or [a-[:digit:]]. It then treats the hyphens as literals. PCRE has no -warning features, so it gives an error in these cases because they are almost -certainly user mistakes. -</P> -<P> -17. In PCRE, the upper/lower case character properties Lu and Ll are not -affected when case-independent matching is specified. For example, \p{Lu} -always matches an upper case letter. I think Perl has changed in this respect; -in the release at the time of writing (5.16), \p{Lu} and \p{Ll} match all -letters, regardless of case, when case independence is specified. -</P> -<P> -18. PCRE provides some extensions to the Perl regular expression facilities. -Perl 5.10 includes new features that are not in earlier versions of Perl, some -of which (such as named parentheses) have been in PCRE for some time. This list -is with respect to Perl 5.10: -<br> -<br> -(a) Although lookbehind assertions in PCRE must match fixed length strings, -each alternative branch of a lookbehind assertion can match a different length -of string. Perl requires them all to have the same length. -<br> -<br> -(b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $ -meta-character matches only at the very end of the string. -<br> -<br> -(c) If PCRE_EXTRA is set, a backslash followed by a letter with no special -meaning is faulted. Otherwise, like Perl, the backslash is quietly ignored. -(Perl can be made to issue a warning.) -<br> -<br> -(d) If PCRE_UNGREEDY is set, the greediness of the repetition quantifiers is -inverted, that is, by default they are not greedy, but if followed by a -question mark they are. -<br> -<br> -(e) PCRE_ANCHORED can be used at matching time to force a pattern to be tried -only at the first matching position in the subject string. -<br> -<br> -(f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, and -PCRE_NO_AUTO_CAPTURE options for <b>pcre_exec()</b> have no Perl equivalents. -<br> -<br> -(g) The \R escape sequence can be restricted to match only CR, LF, or CRLF -by the PCRE_BSR_ANYCRLF option. -<br> -<br> -(h) The callout facility is PCRE-specific. -<br> -<br> -(i) The partial matching facility is PCRE-specific. -<br> -<br> -(j) Patterns compiled by PCRE can be saved and re-used at a later time, even on -different hosts that have the other endianness. However, this does not apply to -optimized data created by the just-in-time compiler. -<br> -<br> -(k) The alternative matching functions (<b>pcre_dfa_exec()</b>, -<b>pcre16_dfa_exec()</b> and <b>pcre32_dfa_exec()</b>,) match in a different way -and are not Perl-compatible. -<br> -<br> -(l) PCRE recognizes some special sequences such as (*CR) at the start of -a pattern that set overall options that cannot be changed within the pattern. -</P> -<br><b> -AUTHOR -</b><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><b> -REVISION -</b><br> -<P> -Last updated: 10 November 2013 -<br> -Copyright © 1997-2013 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcrecpp.html b/plugins/Pcre16/docs/doc/html/pcrecpp.html deleted file mode 100644 index b7eac3a3d7..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcrecpp.html +++ /dev/null @@ -1,368 +0,0 @@ -<html> -<head> -<title>pcrecpp specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcrecpp man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">SYNOPSIS OF C++ WRAPPER</a> -<li><a name="TOC2" href="#SEC2">DESCRIPTION</a> -<li><a name="TOC3" href="#SEC3">MATCHING INTERFACE</a> -<li><a name="TOC4" href="#SEC4">QUOTING METACHARACTERS</a> -<li><a name="TOC5" href="#SEC5">PARTIAL MATCHES</a> -<li><a name="TOC6" href="#SEC6">UTF-8 AND THE MATCHING INTERFACE</a> -<li><a name="TOC7" href="#SEC7">PASSING MODIFIERS TO THE REGULAR EXPRESSION ENGINE</a> -<li><a name="TOC8" href="#SEC8">SCANNING TEXT INCREMENTALLY</a> -<li><a name="TOC9" href="#SEC9">PARSING HEX/OCTAL/C-RADIX NUMBERS</a> -<li><a name="TOC10" href="#SEC10">REPLACING PARTS OF STRINGS</a> -<li><a name="TOC11" href="#SEC11">AUTHOR</a> -<li><a name="TOC12" href="#SEC12">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">SYNOPSIS OF C++ WRAPPER</a><br> -<P> -<b>#include <pcrecpp.h></b> -</P> -<br><a name="SEC2" href="#TOC1">DESCRIPTION</a><br> -<P> -The C++ wrapper for PCRE was provided by Google Inc. Some additional -functionality was added by Giuseppe Maxia. This brief man page was constructed -from the notes in the <i>pcrecpp.h</i> file, which should be consulted for -further details. Note that the C++ wrapper supports only the original 8-bit -PCRE library. There is no 16-bit or 32-bit support at present. -</P> -<br><a name="SEC3" href="#TOC1">MATCHING INTERFACE</a><br> -<P> -The "FullMatch" operation checks that supplied text matches a supplied pattern -exactly. If pointer arguments are supplied, it copies matched sub-strings that -match sub-patterns into them. -<pre> - Example: successful match - pcrecpp::RE re("h.*o"); - re.FullMatch("hello"); - - Example: unsuccessful match (requires full match): - pcrecpp::RE re("e"); - !re.FullMatch("hello"); - - Example: creating a temporary RE object: - pcrecpp::RE("h.*o").FullMatch("hello"); -</pre> -You can pass in a "const char*" or a "string" for "text". The examples below -tend to use a const char*. You can, as in the different examples above, store -the RE object explicitly in a variable or use a temporary RE object. The -examples below use one mode or the other arbitrarily. Either could correctly be -used for any of these examples. -</P> -<P> -You must supply extra pointer arguments to extract matched subpieces. -<pre> - Example: extracts "ruby" into "s" and 1234 into "i" - int i; - string s; - pcrecpp::RE re("(\\w+):(\\d+)"); - re.FullMatch("ruby:1234", &s, &i); - - Example: does not try to extract any extra sub-patterns - re.FullMatch("ruby:1234", &s); - - Example: does not try to extract into NULL - re.FullMatch("ruby:1234", NULL, &i); - - Example: integer overflow causes failure - !re.FullMatch("ruby:1234567891234", NULL, &i); - - Example: fails because there aren't enough sub-patterns: - !pcrecpp::RE("\\w+:\\d+").FullMatch("ruby:1234", &s); - - Example: fails because string cannot be stored in integer - !pcrecpp::RE("(.*)").FullMatch("ruby", &i); -</pre> -The provided pointer arguments can be pointers to any scalar numeric -type, or one of: -<pre> - string (matched piece is copied to string) - StringPiece (StringPiece is mutated to point to matched piece) - T (where "bool T::ParseFrom(const char*, int)" exists) - NULL (the corresponding matched sub-pattern is not copied) -</pre> -The function returns true iff all of the following conditions are satisfied: -<pre> - a. "text" matches "pattern" exactly; - - b. The number of matched sub-patterns is >= number of supplied - pointers; - - c. The "i"th argument has a suitable type for holding the - string captured as the "i"th sub-pattern. If you pass in - void * NULL for the "i"th argument, or a non-void * NULL - of the correct type, or pass fewer arguments than the - number of sub-patterns, "i"th captured sub-pattern is - ignored. -</pre> -CAVEAT: An optional sub-pattern that does not exist in the matched -string is assigned the empty string. Therefore, the following will -return false (because the empty string is not a valid number): -<pre> - int number; - pcrecpp::RE::FullMatch("abc", "[a-z]+(\\d+)?", &number); -</pre> -The matching interface supports at most 16 arguments per call. -If you need more, consider using the more general interface -<b>pcrecpp::RE::DoMatch</b>. See <b>pcrecpp.h</b> for the signature for -<b>DoMatch</b>. -</P> -<P> -NOTE: Do not use <b>no_arg</b>, which is used internally to mark the end of a -list of optional arguments, as a placeholder for missing arguments, as this can -lead to segfaults. -</P> -<br><a name="SEC4" href="#TOC1">QUOTING METACHARACTERS</a><br> -<P> -You can use the "QuoteMeta" operation to insert backslashes before all -potentially meaningful characters in a string. The returned string, used as a -regular expression, will exactly match the original string. -<pre> - Example: - string quoted = RE::QuoteMeta(unquoted); -</pre> -Note that it's legal to escape a character even if it has no special meaning in -a regular expression -- so this function does that. (This also makes it -identical to the perl function of the same name; see "perldoc -f quotemeta".) -For example, "1.5-2.0?" becomes "1\.5\-2\.0\?". -</P> -<br><a name="SEC5" href="#TOC1">PARTIAL MATCHES</a><br> -<P> -You can use the "PartialMatch" operation when you want the pattern -to match any substring of the text. -<pre> - Example: simple search for a string: - pcrecpp::RE("ell").PartialMatch("hello"); - - Example: find first number in a string: - int number; - pcrecpp::RE re("(\\d+)"); - re.PartialMatch("x*100 + 20", &number); - assert(number == 100); -</PRE> -</P> -<br><a name="SEC6" href="#TOC1">UTF-8 AND THE MATCHING INTERFACE</a><br> -<P> -By default, pattern and text are plain text, one byte per character. The UTF8 -flag, passed to the constructor, causes both pattern and string to be treated -as UTF-8 text, still a byte stream but potentially multiple bytes per -character. In practice, the text is likelier to be UTF-8 than the pattern, but -the match returned may depend on the UTF8 flag, so always use it when matching -UTF8 text. For example, "." will match one byte normally but with UTF8 set may -match up to three bytes of a multi-byte character. -<pre> - Example: - pcrecpp::RE_Options options; - options.set_utf8(); - pcrecpp::RE re(utf8_pattern, options); - re.FullMatch(utf8_string); - - Example: using the convenience function UTF8(): - pcrecpp::RE re(utf8_pattern, pcrecpp::UTF8()); - re.FullMatch(utf8_string); -</pre> -NOTE: The UTF8 flag is ignored if pcre was not configured with the -<pre> - --enable-utf8 flag. -</PRE> -</P> -<br><a name="SEC7" href="#TOC1">PASSING MODIFIERS TO THE REGULAR EXPRESSION ENGINE</a><br> -<P> -PCRE defines some modifiers to change the behavior of the regular expression -engine. The C++ wrapper defines an auxiliary class, RE_Options, as a vehicle to -pass such modifiers to a RE class. Currently, the following modifiers are -supported: -<pre> - modifier description Perl corresponding - - PCRE_CASELESS case insensitive match /i - PCRE_MULTILINE multiple lines match /m - PCRE_DOTALL dot matches newlines /s - PCRE_DOLLAR_ENDONLY $ matches only at end N/A - PCRE_EXTRA strict escape parsing N/A - PCRE_EXTENDED ignore white spaces /x - PCRE_UTF8 handles UTF8 chars built-in - PCRE_UNGREEDY reverses * and *? N/A - PCRE_NO_AUTO_CAPTURE disables capturing parens N/A (*) -</pre> -(*) Both Perl and PCRE allow non capturing parentheses by means of the -"?:" modifier within the pattern itself. e.g. (?:ab|cd) does not -capture, while (ab|cd) does. -</P> -<P> -For a full account on how each modifier works, please check the -PCRE API reference page. -</P> -<P> -For each modifier, there are two member functions whose name is made -out of the modifier in lowercase, without the "PCRE_" prefix. For -instance, PCRE_CASELESS is handled by -<pre> - bool caseless() -</pre> -which returns true if the modifier is set, and -<pre> - RE_Options & set_caseless(bool) -</pre> -which sets or unsets the modifier. Moreover, PCRE_EXTRA_MATCH_LIMIT can be -accessed through the <b>set_match_limit()</b> and <b>match_limit()</b> member -functions. Setting <i>match_limit</i> to a non-zero value will limit the -execution of pcre to keep it from doing bad things like blowing the stack or -taking an eternity to return a result. A value of 5000 is good enough to stop -stack blowup in a 2MB thread stack. Setting <i>match_limit</i> to zero disables -match limiting. Alternatively, you can call <b>match_limit_recursion()</b> -which uses PCRE_EXTRA_MATCH_LIMIT_RECURSION to limit how much PCRE -recurses. <b>match_limit()</b> limits the number of matches PCRE does; -<b>match_limit_recursion()</b> limits the depth of internal recursion, and -therefore the amount of stack that is used. -</P> -<P> -Normally, to pass one or more modifiers to a RE class, you declare -a <i>RE_Options</i> object, set the appropriate options, and pass this -object to a RE constructor. Example: -<pre> - RE_Options opt; - opt.set_caseless(true); - if (RE("HELLO", opt).PartialMatch("hello world")) ... -</pre> -RE_options has two constructors. The default constructor takes no arguments and -creates a set of flags that are off by default. The optional parameter -<i>option_flags</i> is to facilitate transfer of legacy code from C programs. -This lets you do -<pre> - RE(pattern, - RE_Options(PCRE_CASELESS|PCRE_MULTILINE)).PartialMatch(str); -</pre> -However, new code is better off doing -<pre> - RE(pattern, - RE_Options().set_caseless(true).set_multiline(true)) - .PartialMatch(str); -</pre> -If you are going to pass one of the most used modifiers, there are some -convenience functions that return a RE_Options class with the -appropriate modifier already set: <b>CASELESS()</b>, <b>UTF8()</b>, -<b>MULTILINE()</b>, <b>DOTALL</b>(), and <b>EXTENDED()</b>. -</P> -<P> -If you need to set several options at once, and you don't want to go through -the pains of declaring a RE_Options object and setting several options, there -is a parallel method that give you such ability on the fly. You can concatenate -several <b>set_xxxxx()</b> member functions, since each of them returns a -reference to its class object. For example, to pass PCRE_CASELESS, -PCRE_EXTENDED, and PCRE_MULTILINE to a RE with one statement, you may write: -<pre> - RE(" ^ xyz \\s+ .* blah$", - RE_Options() - .set_caseless(true) - .set_extended(true) - .set_multiline(true)).PartialMatch(sometext); - -</PRE> -</P> -<br><a name="SEC8" href="#TOC1">SCANNING TEXT INCREMENTALLY</a><br> -<P> -The "Consume" operation may be useful if you want to repeatedly -match regular expressions at the front of a string and skip over -them as they match. This requires use of the "StringPiece" type, -which represents a sub-range of a real string. Like RE, StringPiece -is defined in the pcrecpp namespace. -<pre> - Example: read lines of the form "var = value" from a string. - string contents = ...; // Fill string somehow - pcrecpp::StringPiece input(contents); // Wrap in a StringPiece - - string var; - int value; - pcrecpp::RE re("(\\w+) = (\\d+)\n"); - while (re.Consume(&input, &var, &value)) { - ...; - } -</pre> -Each successful call to "Consume" will set "var/value", and also -advance "input" so it points past the matched text. -</P> -<P> -The "FindAndConsume" operation is similar to "Consume" but does not -anchor your match at the beginning of the string. For example, you -could extract all words from a string by repeatedly calling -<pre> - pcrecpp::RE("(\\w+)").FindAndConsume(&input, &word) -</PRE> -</P> -<br><a name="SEC9" href="#TOC1">PARSING HEX/OCTAL/C-RADIX NUMBERS</a><br> -<P> -By default, if you pass a pointer to a numeric value, the -corresponding text is interpreted as a base-10 number. You can -instead wrap the pointer with a call to one of the operators Hex(), -Octal(), or CRadix() to interpret the text in another base. The -CRadix operator interprets C-style "0" (base-8) and "0x" (base-16) -prefixes, but defaults to base-10. -<pre> - Example: - int a, b, c, d; - pcrecpp::RE re("(.*) (.*) (.*) (.*)"); - re.FullMatch("100 40 0100 0x40", - pcrecpp::Octal(&a), pcrecpp::Hex(&b), - pcrecpp::CRadix(&c), pcrecpp::CRadix(&d)); -</pre> -will leave 64 in a, b, c, and d. -</P> -<br><a name="SEC10" href="#TOC1">REPLACING PARTS OF STRINGS</a><br> -<P> -You can replace the first match of "pattern" in "str" with "rewrite". -Within "rewrite", backslash-escaped digits (\1 to \9) can be -used to insert text matching corresponding parenthesized group -from the pattern. \0 in "rewrite" refers to the entire matching -text. For example: -<pre> - string s = "yabba dabba doo"; - pcrecpp::RE("b+").Replace("d", &s); -</pre> -will leave "s" containing "yada dabba doo". The result is true if the pattern -matches and a replacement occurs, false otherwise. -</P> -<P> -<b>GlobalReplace</b> is like <b>Replace</b> except that it replaces all -occurrences of the pattern in the string with the rewrite. Replacements are -not subject to re-matching. For example: -<pre> - string s = "yabba dabba doo"; - pcrecpp::RE("b+").GlobalReplace("d", &s); -</pre> -will leave "s" containing "yada dada doo". It returns the number of -replacements made. -</P> -<P> -<b>Extract</b> is like <b>Replace</b>, except that if the pattern matches, -"rewrite" is copied into "out" (an additional argument) with substitutions. -The non-matching portions of "text" are ignored. Returns true iff a match -occurred and the extraction happened successfully; if no match occurs, the -string is left unaffected. -</P> -<br><a name="SEC11" href="#TOC1">AUTHOR</a><br> -<P> -The C++ wrapper was contributed by Google Inc. -<br> -Copyright © 2007 Google Inc. -<br> -</P> -<br><a name="SEC12" href="#TOC1">REVISION</a><br> -<P> -Last updated: 08 January 2012 -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcredemo.html b/plugins/Pcre16/docs/doc/html/pcredemo.html deleted file mode 100644 index 894a930826..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcredemo.html +++ /dev/null @@ -1,426 +0,0 @@ -<html> -<head> -<title>pcredemo specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcredemo man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -</ul> -<PRE> -/************************************************* -* PCRE DEMONSTRATION PROGRAM * -*************************************************/ - -/* This is a demonstration program to illustrate the most straightforward ways -of calling the PCRE regular expression library from a C program. See the -pcresample documentation for a short discussion ("man pcresample" if you have -the PCRE man pages installed). - -In Unix-like environments, if PCRE is installed in your standard system -libraries, you should be able to compile this program using this command: - -gcc -Wall pcredemo.c -lpcre -o pcredemo - -If PCRE is not installed in a standard place, it is likely to be installed with -support for the pkg-config mechanism. If you have pkg-config, you can compile -this program using this command: - -gcc -Wall pcredemo.c `pkg-config --cflags --libs libpcre` -o pcredemo - -If you do not have pkg-config, you may have to use this: - -gcc -Wall pcredemo.c -I/usr/local/include -L/usr/local/lib \ - -R/usr/local/lib -lpcre -o pcredemo - -Replace "/usr/local/include" and "/usr/local/lib" with wherever the include and -library files for PCRE are installed on your system. Only some operating -systems (e.g. Solaris) use the -R option. - -Building under Windows: - -If you want to statically link this program against a non-dll .a file, you must -define PCRE_STATIC before including pcre.h, otherwise the pcre_malloc() and -pcre_free() exported functions will be declared __declspec(dllimport), with -unwanted results. So in this environment, uncomment the following line. */ - -/* #define PCRE_STATIC */ - -#include <stdio.h> -#include <string.h> -#include <pcre.h> - -#define OVECCOUNT 30 /* should be a multiple of 3 */ - - -int main(int argc, char **argv) -{ -pcre *re; -const char *error; -char *pattern; -char *subject; -unsigned char *name_table; -unsigned int option_bits; -int erroffset; -int find_all; -int crlf_is_newline; -int namecount; -int name_entry_size; -int ovector[OVECCOUNT]; -int subject_length; -int rc, i; -int utf8; - - -/************************************************************************** -* First, sort out the command line. There is only one possible option at * -* the moment, "-g" to request repeated matching to find all occurrences, * -* like Perl's /g option. We set the variable find_all to a non-zero value * -* if the -g option is present. Apart from that, there must be exactly two * -* arguments. * -**************************************************************************/ - -find_all = 0; -for (i = 1; i < argc; i++) - { - if (strcmp(argv[i], "-g") == 0) find_all = 1; - else break; - } - -/* After the options, we require exactly two arguments, which are the pattern, -and the subject string. */ - -if (argc - i != 2) - { - printf("Two arguments required: a regex and a subject string\n"); - return 1; - } - -pattern = argv[i]; -subject = argv[i+1]; -subject_length = (int)strlen(subject); - - -/************************************************************************* -* Now we are going to compile the regular expression pattern, and handle * -* and errors that are detected. * -*************************************************************************/ - -re = pcre_compile( - pattern, /* the pattern */ - 0, /* default options */ - &error, /* for error message */ - &erroffset, /* for error offset */ - NULL); /* use default character tables */ - -/* Compilation failed: print the error message and exit */ - -if (re == NULL) - { - printf("PCRE compilation failed at offset %d: %s\n", erroffset, error); - return 1; - } - - -/************************************************************************* -* If the compilation succeeded, we call PCRE again, in order to do a * -* pattern match against the subject string. This does just ONE match. If * -* further matching is needed, it will be done below. * -*************************************************************************/ - -rc = pcre_exec( - re, /* the compiled pattern */ - NULL, /* no extra data - we didn't study the pattern */ - subject, /* the subject string */ - subject_length, /* the length of the subject */ - 0, /* start at offset 0 in the subject */ - 0, /* default options */ - ovector, /* output vector for substring information */ - OVECCOUNT); /* number of elements in the output vector */ - -/* Matching failed: handle error cases */ - -if (rc < 0) - { - switch(rc) - { - case PCRE_ERROR_NOMATCH: printf("No match\n"); break; - /* - Handle other special cases if you like - */ - default: printf("Matching error %d\n", rc); break; - } - pcre_free(re); /* Release memory used for the compiled pattern */ - return 1; - } - -/* Match succeded */ - -printf("\nMatch succeeded at offset %d\n", ovector[0]); - - -/************************************************************************* -* We have found the first match within the subject string. If the output * -* vector wasn't big enough, say so. Then output any substrings that were * -* captured. * -*************************************************************************/ - -/* The output vector wasn't big enough */ - -if (rc == 0) - { - rc = OVECCOUNT/3; - printf("ovector only has room for %d captured substrings\n", rc - 1); - } - -/* Show substrings stored in the output vector by number. Obviously, in a real -application you might want to do things other than print them. */ - -for (i = 0; i < rc; i++) - { - char *substring_start = subject + ovector[2*i]; - int substring_length = ovector[2*i+1] - ovector[2*i]; - printf("%2d: %.*s\n", i, substring_length, substring_start); - } - - -/************************************************************************** -* That concludes the basic part of this demonstration program. We have * -* compiled a pattern, and performed a single match. The code that follows * -* shows first how to access named substrings, and then how to code for * -* repeated matches on the same subject. * -**************************************************************************/ - -/* See if there are any named substrings, and if so, show them by name. First -we have to extract the count of named parentheses from the pattern. */ - -(void)pcre_fullinfo( - re, /* the compiled pattern */ - NULL, /* no extra data - we didn't study the pattern */ - PCRE_INFO_NAMECOUNT, /* number of named substrings */ - &namecount); /* where to put the answer */ - -if (namecount <= 0) printf("No named substrings\n"); else - { - unsigned char *tabptr; - printf("Named substrings\n"); - - /* Before we can access the substrings, we must extract the table for - translating names to numbers, and the size of each entry in the table. */ - - (void)pcre_fullinfo( - re, /* the compiled pattern */ - NULL, /* no extra data - we didn't study the pattern */ - PCRE_INFO_NAMETABLE, /* address of the table */ - &name_table); /* where to put the answer */ - - (void)pcre_fullinfo( - re, /* the compiled pattern */ - NULL, /* no extra data - we didn't study the pattern */ - PCRE_INFO_NAMEENTRYSIZE, /* size of each entry in the table */ - &name_entry_size); /* where to put the answer */ - - /* Now we can scan the table and, for each entry, print the number, the name, - and the substring itself. */ - - tabptr = name_table; - for (i = 0; i < namecount; i++) - { - int n = (tabptr[0] << 8) | tabptr[1]; - printf("(%d) %*s: %.*s\n", n, name_entry_size - 3, tabptr + 2, - ovector[2*n+1] - ovector[2*n], subject + ovector[2*n]); - tabptr += name_entry_size; - } - } - - -/************************************************************************* -* If the "-g" option was given on the command line, we want to continue * -* to search for additional matches in the subject string, in a similar * -* way to the /g option in Perl. This turns out to be trickier than you * -* might think because of the possibility of matching an empty string. * -* What happens is as follows: * -* * -* If the previous match was NOT for an empty string, we can just start * -* the next match at the end of the previous one. * -* * -* If the previous match WAS for an empty string, we can't do that, as it * -* would lead to an infinite loop. Instead, a special call of pcre_exec() * -* is made with the PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED flags set. * -* The first of these tells PCRE that an empty string at the start of the * -* subject is not a valid match; other possibilities must be tried. The * -* second flag restricts PCRE to one match attempt at the initial string * -* position. If this match succeeds, an alternative to the empty string * -* match has been found, and we can print it and proceed round the loop, * -* advancing by the length of whatever was found. If this match does not * -* succeed, we still stay in the loop, advancing by just one character. * -* In UTF-8 mode, which can be set by (*UTF8) in the pattern, this may be * -* more than one byte. * -* * -* However, there is a complication concerned with newlines. When the * -* newline convention is such that CRLF is a valid newline, we must * -* advance by two characters rather than one. The newline convention can * -* be set in the regex by (*CR), etc.; if not, we must find the default. * -*************************************************************************/ - -if (!find_all) /* Check for -g */ - { - pcre_free(re); /* Release the memory used for the compiled pattern */ - return 0; /* Finish unless -g was given */ - } - -/* Before running the loop, check for UTF-8 and whether CRLF is a valid newline -sequence. First, find the options with which the regex was compiled; extract -the UTF-8 state, and mask off all but the newline options. */ - -(void)pcre_fullinfo(re, NULL, PCRE_INFO_OPTIONS, &option_bits); -utf8 = option_bits & PCRE_UTF8; -option_bits &= PCRE_NEWLINE_CR|PCRE_NEWLINE_LF|PCRE_NEWLINE_CRLF| - PCRE_NEWLINE_ANY|PCRE_NEWLINE_ANYCRLF; - -/* If no newline options were set, find the default newline convention from the -build configuration. */ - -if (option_bits == 0) - { - int d; - (void)pcre_config(PCRE_CONFIG_NEWLINE, &d); - /* Note that these values are always the ASCII ones, even in - EBCDIC environments. CR = 13, NL = 10. */ - option_bits = (d == 13)? PCRE_NEWLINE_CR : - (d == 10)? PCRE_NEWLINE_LF : - (d == (13<<8 | 10))? PCRE_NEWLINE_CRLF : - (d == -2)? PCRE_NEWLINE_ANYCRLF : - (d == -1)? PCRE_NEWLINE_ANY : 0; - } - -/* See if CRLF is a valid newline sequence. */ - -crlf_is_newline = - option_bits == PCRE_NEWLINE_ANY || - option_bits == PCRE_NEWLINE_CRLF || - option_bits == PCRE_NEWLINE_ANYCRLF; - -/* Loop for second and subsequent matches */ - -for (;;) - { - int options = 0; /* Normally no options */ - int start_offset = ovector[1]; /* Start at end of previous match */ - - /* If the previous match was for an empty string, we are finished if we are - at the end of the subject. Otherwise, arrange to run another match at the - same point to see if a non-empty match can be found. */ - - if (ovector[0] == ovector[1]) - { - if (ovector[0] == subject_length) break; - options = PCRE_NOTEMPTY_ATSTART | PCRE_ANCHORED; - } - - /* Run the next matching operation */ - - rc = pcre_exec( - re, /* the compiled pattern */ - NULL, /* no extra data - we didn't study the pattern */ - subject, /* the subject string */ - subject_length, /* the length of the subject */ - start_offset, /* starting offset in the subject */ - options, /* options */ - ovector, /* output vector for substring information */ - OVECCOUNT); /* number of elements in the output vector */ - - /* This time, a result of NOMATCH isn't an error. If the value in "options" - is zero, it just means we have found all possible matches, so the loop ends. - Otherwise, it means we have failed to find a non-empty-string match at a - point where there was a previous empty-string match. In this case, we do what - Perl does: advance the matching position by one character, and continue. We - do this by setting the "end of previous match" offset, because that is picked - up at the top of the loop as the point at which to start again. - - There are two complications: (a) When CRLF is a valid newline sequence, and - the current position is just before it, advance by an extra byte. (b) - Otherwise we must ensure that we skip an entire UTF-8 character if we are in - UTF-8 mode. */ - - if (rc == PCRE_ERROR_NOMATCH) - { - if (options == 0) break; /* All matches found */ - ovector[1] = start_offset + 1; /* Advance one byte */ - if (crlf_is_newline && /* If CRLF is newline & */ - start_offset < subject_length - 1 && /* we are at CRLF, */ - subject[start_offset] == '\r' && - subject[start_offset + 1] == '\n') - ovector[1] += 1; /* Advance by one more. */ - else if (utf8) /* Otherwise, ensure we */ - { /* advance a whole UTF-8 */ - while (ovector[1] < subject_length) /* character. */ - { - if ((subject[ovector[1]] & 0xc0) != 0x80) break; - ovector[1] += 1; - } - } - continue; /* Go round the loop again */ - } - - /* Other matching errors are not recoverable. */ - - if (rc < 0) - { - printf("Matching error %d\n", rc); - pcre_free(re); /* Release memory used for the compiled pattern */ - return 1; - } - - /* Match succeded */ - - printf("\nMatch succeeded again at offset %d\n", ovector[0]); - - /* The match succeeded, but the output vector wasn't big enough. */ - - if (rc == 0) - { - rc = OVECCOUNT/3; - printf("ovector only has room for %d captured substrings\n", rc - 1); - } - - /* As before, show substrings stored in the output vector by number, and then - also any named substrings. */ - - for (i = 0; i < rc; i++) - { - char *substring_start = subject + ovector[2*i]; - int substring_length = ovector[2*i+1] - ovector[2*i]; - printf("%2d: %.*s\n", i, substring_length, substring_start); - } - - if (namecount <= 0) printf("No named substrings\n"); else - { - unsigned char *tabptr = name_table; - printf("Named substrings\n"); - for (i = 0; i < namecount; i++) - { - int n = (tabptr[0] << 8) | tabptr[1]; - printf("(%d) %*s: %.*s\n", n, name_entry_size - 3, tabptr + 2, - ovector[2*n+1] - ovector[2*n], subject + ovector[2*n]); - tabptr += name_entry_size; - } - } - } /* End of loop to find second and subsequent matches */ - -printf("\n"); -pcre_free(re); /* Release memory used for the compiled pattern */ -return 0; -} - -/* End of pcredemo.c */ -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcregrep.html b/plugins/Pcre16/docs/doc/html/pcregrep.html deleted file mode 100644 index dacbb4998f..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcregrep.html +++ /dev/null @@ -1,759 +0,0 @@ -<html> -<head> -<title>pcregrep specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcregrep man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">SYNOPSIS</a> -<li><a name="TOC2" href="#SEC2">DESCRIPTION</a> -<li><a name="TOC3" href="#SEC3">SUPPORT FOR COMPRESSED FILES</a> -<li><a name="TOC4" href="#SEC4">BINARY FILES</a> -<li><a name="TOC5" href="#SEC5">OPTIONS</a> -<li><a name="TOC6" href="#SEC6">ENVIRONMENT VARIABLES</a> -<li><a name="TOC7" href="#SEC7">NEWLINES</a> -<li><a name="TOC8" href="#SEC8">OPTIONS COMPATIBILITY</a> -<li><a name="TOC9" href="#SEC9">OPTIONS WITH DATA</a> -<li><a name="TOC10" href="#SEC10">MATCHING ERRORS</a> -<li><a name="TOC11" href="#SEC11">DIAGNOSTICS</a> -<li><a name="TOC12" href="#SEC12">SEE ALSO</a> -<li><a name="TOC13" href="#SEC13">AUTHOR</a> -<li><a name="TOC14" href="#SEC14">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">SYNOPSIS</a><br> -<P> -<b>pcregrep [options] [long options] [pattern] [path1 path2 ...]</b> -</P> -<br><a name="SEC2" href="#TOC1">DESCRIPTION</a><br> -<P> -<b>pcregrep</b> searches files for character patterns, in the same way as other -grep commands do, but it uses the PCRE regular expression library to support -patterns that are compatible with the regular expressions of Perl 5. See -<a href="pcresyntax.html"><b>pcresyntax</b>(3)</a> -for a quick-reference summary of pattern syntax, or -<a href="pcrepattern.html"><b>pcrepattern</b>(3)</a> -for a full description of the syntax and semantics of the regular expressions -that PCRE supports. -</P> -<P> -Patterns, whether supplied on the command line or in a separate file, are given -without delimiters. For example: -<pre> - pcregrep Thursday /etc/motd -</pre> -If you attempt to use delimiters (for example, by surrounding a pattern with -slashes, as is common in Perl scripts), they are interpreted as part of the -pattern. Quotes can of course be used to delimit patterns on the command line -because they are interpreted by the shell, and indeed quotes are required if a -pattern contains white space or shell metacharacters. -</P> -<P> -The first argument that follows any option settings is treated as the single -pattern to be matched when neither <b>-e</b> nor <b>-f</b> is present. -Conversely, when one or both of these options are used to specify patterns, all -arguments are treated as path names. At least one of <b>-e</b>, <b>-f</b>, or an -argument pattern must be provided. -</P> -<P> -If no files are specified, <b>pcregrep</b> reads the standard input. The -standard input can also be referenced by a name consisting of a single hyphen. -For example: -<pre> - pcregrep some-pattern /file1 - /file3 -</pre> -By default, each line that matches a pattern is copied to the standard -output, and if there is more than one file, the file name is output at the -start of each line, followed by a colon. However, there are options that can -change how <b>pcregrep</b> behaves. In particular, the <b>-M</b> option makes it -possible to search for patterns that span line boundaries. What defines a line -boundary is controlled by the <b>-N</b> (<b>--newline</b>) option. -</P> -<P> -The amount of memory used for buffering files that are being scanned is -controlled by a parameter that can be set by the <b>--buffer-size</b> option. -The default value for this parameter is specified when <b>pcregrep</b> is built, -with the default default being 20K. A block of memory three times this size is -used (to allow for buffering "before" and "after" lines). An error occurs if a -line overflows the buffer. -</P> -<P> -Patterns can be no longer than 8K or BUFSIZ bytes, whichever is the greater. -BUFSIZ is defined in <b><stdio.h></b>. When there is more than one pattern -(specified by the use of <b>-e</b> and/or <b>-f</b>), each pattern is applied to -each line in the order in which they are defined, except that all the <b>-e</b> -patterns are tried before the <b>-f</b> patterns. -</P> -<P> -By default, as soon as one pattern matches a line, no further patterns are -considered. However, if <b>--colour</b> (or <b>--color</b>) is used to colour the -matching substrings, or if <b>--only-matching</b>, <b>--file-offsets</b>, or -<b>--line-offsets</b> is used to output only the part of the line that matched -(either shown literally, or as an offset), scanning resumes immediately -following the match, so that further matches on the same line can be found. If -there are multiple patterns, they are all tried on the remainder of the line, -but patterns that follow the one that matched are not tried on the earlier part -of the line. -</P> -<P> -This behaviour means that the order in which multiple patterns are specified -can affect the output when one of the above options is used. This is no longer -the same behaviour as GNU grep, which now manages to display earlier matches -for later patterns (as long as there is no overlap). -</P> -<P> -Patterns that can match an empty string are accepted, but empty string -matches are never recognized. An example is the pattern "(super)?(man)?", in -which all components are optional. This pattern finds all occurrences of both -"super" and "man"; the output differs from matching with "super|man" when only -the matching substrings are being shown. -</P> -<P> -If the <b>LC_ALL</b> or <b>LC_CTYPE</b> environment variable is set, -<b>pcregrep</b> uses the value to set a locale when calling the PCRE library. -The <b>--locale</b> option can be used to override this. -</P> -<br><a name="SEC3" href="#TOC1">SUPPORT FOR COMPRESSED FILES</a><br> -<P> -It is possible to compile <b>pcregrep</b> so that it uses <b>libz</b> or -<b>libbz2</b> to read files whose names end in <b>.gz</b> or <b>.bz2</b>, -respectively. You can find out whether your binary has support for one or both -of these file types by running it with the <b>--help</b> option. If the -appropriate support is not present, files are treated as plain text. The -standard input is always so treated. -</P> -<br><a name="SEC4" href="#TOC1">BINARY FILES</a><br> -<P> -By default, a file that contains a binary zero byte within the first 1024 bytes -is identified as a binary file, and is processed specially. (GNU grep also -identifies binary files in this manner.) See the <b>--binary-files</b> option -for a means of changing the way binary files are handled. -</P> -<br><a name="SEC5" href="#TOC1">OPTIONS</a><br> -<P> -The order in which some of the options appear can affect the output. For -example, both the <b>-h</b> and <b>-l</b> options affect the printing of file -names. Whichever comes later in the command line will be the one that takes -effect. Similarly, except where noted below, if an option is given twice, the -later setting is used. Numerical values for options may be followed by K or M, -to signify multiplication by 1024 or 1024*1024 respectively. -</P> -<P> -<b>--</b> -This terminates the list of options. It is useful if the next item on the -command line starts with a hyphen but is not an option. This allows for the -processing of patterns and filenames that start with hyphens. -</P> -<P> -<b>-A</b> <i>number</i>, <b>--after-context=</b><i>number</i> -Output <i>number</i> lines of context after each matching line. If filenames -and/or line numbers are being output, a hyphen separator is used instead of a -colon for the context lines. A line containing "--" is output between each -group of lines, unless they are in fact contiguous in the input file. The value -of <i>number</i> is expected to be relatively small. However, <b>pcregrep</b> -guarantees to have up to 8K of following text available for context output. -</P> -<P> -<b>-a</b>, <b>--text</b> -Treat binary files as text. This is equivalent to -<b>--binary-files</b>=<i>text</i>. -</P> -<P> -<b>-B</b> <i>number</i>, <b>--before-context=</b><i>number</i> -Output <i>number</i> lines of context before each matching line. If filenames -and/or line numbers are being output, a hyphen separator is used instead of a -colon for the context lines. A line containing "--" is output between each -group of lines, unless they are in fact contiguous in the input file. The value -of <i>number</i> is expected to be relatively small. However, <b>pcregrep</b> -guarantees to have up to 8K of preceding text available for context output. -</P> -<P> -<b>--binary-files=</b><i>word</i> -Specify how binary files are to be processed. If the word is "binary" (the -default), pattern matching is performed on binary files, but the only output is -"Binary file <name> matches" when a match succeeds. If the word is "text", -which is equivalent to the <b>-a</b> or <b>--text</b> option, binary files are -processed in the same way as any other file. In this case, when a match -succeeds, the output may be binary garbage, which can have nasty effects if -sent to a terminal. If the word is "without-match", which is equivalent to the -<b>-I</b> option, binary files are not processed at all; they are assumed not to -be of interest. -</P> -<P> -<b>--buffer-size=</b><i>number</i> -Set the parameter that controls how much memory is used for buffering files -that are being scanned. -</P> -<P> -<b>-C</b> <i>number</i>, <b>--context=</b><i>number</i> -Output <i>number</i> lines of context both before and after each matching line. -This is equivalent to setting both <b>-A</b> and <b>-B</b> to the same value. -</P> -<P> -<b>-c</b>, <b>--count</b> -Do not output individual lines from the files that are being scanned; instead -output the number of lines that would otherwise have been shown. If no lines -are selected, the number zero is output. If several files are are being -scanned, a count is output for each of them. However, if the -<b>--files-with-matches</b> option is also used, only those files whose counts -are greater than zero are listed. When <b>-c</b> is used, the <b>-A</b>, -<b>-B</b>, and <b>-C</b> options are ignored. -</P> -<P> -<b>--colour</b>, <b>--color</b> -If this option is given without any data, it is equivalent to "--colour=auto". -If data is required, it must be given in the same shell item, separated by an -equals sign. -</P> -<P> -<b>--colour=</b><i>value</i>, <b>--color=</b><i>value</i> -This option specifies under what circumstances the parts of a line that matched -a pattern should be coloured in the output. By default, the output is not -coloured. The value (which is optional, see above) may be "never", "always", or -"auto". In the latter case, colouring happens only if the standard output is -connected to a terminal. More resources are used when colouring is enabled, -because <b>pcregrep</b> has to search for all possible matches in a line, not -just one, in order to colour them all. -<br> -<br> -The colour that is used can be specified by setting the environment variable -PCREGREP_COLOUR or PCREGREP_COLOR. The value of this variable should be a -string of two numbers, separated by a semicolon. They are copied directly into -the control string for setting colour on a terminal, so it is your -responsibility to ensure that they make sense. If neither of the environment -variables is set, the default is "1;31", which gives red. -</P> -<P> -<b>-D</b> <i>action</i>, <b>--devices=</b><i>action</i> -If an input path is not a regular file or a directory, "action" specifies how -it is to be processed. Valid values are "read" (the default) or "skip" -(silently skip the path). -</P> -<P> -<b>-d</b> <i>action</i>, <b>--directories=</b><i>action</i> -If an input path is a directory, "action" specifies how it is to be processed. -Valid values are "read" (the default in non-Windows environments, for -compatibility with GNU grep), "recurse" (equivalent to the <b>-r</b> option), or -"skip" (silently skip the path, the default in Windows environments). In the -"read" case, directories are read as if they were ordinary files. In some -operating systems the effect of reading a directory like this is an immediate -end-of-file; in others it may provoke an error. -</P> -<P> -<b>-e</b> <i>pattern</i>, <b>--regex=</b><i>pattern</i>, <b>--regexp=</b><i>pattern</i> -Specify a pattern to be matched. This option can be used multiple times in -order to specify several patterns. It can also be used as a way of specifying a -single pattern that starts with a hyphen. When <b>-e</b> is used, no argument -pattern is taken from the command line; all arguments are treated as file -names. There is no limit to the number of patterns. They are applied to each -line in the order in which they are defined until one matches. -<br> -<br> -If <b>-f</b> is used with <b>-e</b>, the command line patterns are matched first, -followed by the patterns from the file(s), independent of the order in which -these options are specified. Note that multiple use of <b>-e</b> is not the same -as a single pattern with alternatives. For example, X|Y finds the first -character in a line that is X or Y, whereas if the two patterns are given -separately, with X first, <b>pcregrep</b> finds X if it is present, even if it -follows Y in the line. It finds Y only if there is no X in the line. This -matters only if you are using <b>-o</b> or <b>--colo(u)r</b> to show the part(s) -of the line that matched. -</P> -<P> -<b>--exclude</b>=<i>pattern</i> -Files (but not directories) whose names match the pattern are skipped without -being processed. This applies to all files, whether listed on the command line, -obtained from <b>--file-list</b>, or by scanning a directory. The pattern is a -PCRE regular expression, and is matched against the final component of the file -name, not the entire path. The <b>-F</b>, <b>-w</b>, and <b>-x</b> options do not -apply to this pattern. The option may be given any number of times in order to -specify multiple patterns. If a file name matches both an <b>--include</b> -and an <b>--exclude</b> pattern, it is excluded. There is no short form for this -option. -</P> -<P> -<b>--exclude-from=</b><i>filename</i> -Treat each non-empty line of the file as the data for an <b>--exclude</b> -option. What constitutes a newline when reading the file is the operating -system's default. The <b>--newline</b> option has no effect on this option. This -option may be given more than once in order to specify a number of files to -read. -</P> -<P> -<b>--exclude-dir</b>=<i>pattern</i> -Directories whose names match the pattern are skipped without being processed, -whatever the setting of the <b>--recursive</b> option. This applies to all -directories, whether listed on the command line, obtained from -<b>--file-list</b>, or by scanning a parent directory. The pattern is a PCRE -regular expression, and is matched against the final component of the directory -name, not the entire path. The <b>-F</b>, <b>-w</b>, and <b>-x</b> options do not -apply to this pattern. The option may be given any number of times in order to -specify more than one pattern. If a directory matches both <b>--include-dir</b> -and <b>--exclude-dir</b>, it is excluded. There is no short form for this -option. -</P> -<P> -<b>-F</b>, <b>--fixed-strings</b> -Interpret each data-matching pattern as a list of fixed strings, separated by -newlines, instead of as a regular expression. What constitutes a newline for -this purpose is controlled by the <b>--newline</b> option. The <b>-w</b> (match -as a word) and <b>-x</b> (match whole line) options can be used with <b>-F</b>. -They apply to each of the fixed strings. A line is selected if any of the fixed -strings are found in it (subject to <b>-w</b> or <b>-x</b>, if present). This -option applies only to the patterns that are matched against the contents of -files; it does not apply to patterns specified by any of the <b>--include</b> or -<b>--exclude</b> options. -</P> -<P> -<b>-f</b> <i>filename</i>, <b>--file=</b><i>filename</i> -Read patterns from the file, one per line, and match them against -each line of input. What constitutes a newline when reading the file is the -operating system's default. The <b>--newline</b> option has no effect on this -option. Trailing white space is removed from each line, and blank lines are -ignored. An empty file contains no patterns and therefore matches nothing. See -also the comments about multiple patterns versus a single pattern with -alternatives in the description of <b>-e</b> above. -<br> -<br> -If this option is given more than once, all the specified files are -read. A data line is output if any of the patterns match it. A filename can -be given as "-" to refer to the standard input. When <b>-f</b> is used, patterns -specified on the command line using <b>-e</b> may also be present; they are -tested before the file's patterns. However, no other pattern is taken from the -command line; all arguments are treated as the names of paths to be searched. -</P> -<P> -<b>--file-list</b>=<i>filename</i> -Read a list of files and/or directories that are to be scanned from the given -file, one per line. Trailing white space is removed from each line, and blank -lines are ignored. These paths are processed before any that are listed on the -command line. The filename can be given as "-" to refer to the standard input. -If <b>--file</b> and <b>--file-list</b> are both specified as "-", patterns are -read first. This is useful only when the standard input is a terminal, from -which further lines (the list of files) can be read after an end-of-file -indication. If this option is given more than once, all the specified files are -read. -</P> -<P> -<b>--file-offsets</b> -Instead of showing lines or parts of lines that match, show each match as an -offset from the start of the file and a length, separated by a comma. In this -mode, no context is shown. That is, the <b>-A</b>, <b>-B</b>, and <b>-C</b> -options are ignored. If there is more than one match in a line, each of them is -shown separately. This option is mutually exclusive with <b>--line-offsets</b> -and <b>--only-matching</b>. -</P> -<P> -<b>-H</b>, <b>--with-filename</b> -Force the inclusion of the filename at the start of output lines when searching -a single file. By default, the filename is not shown in this case. For matching -lines, the filename is followed by a colon; for context lines, a hyphen -separator is used. If a line number is also being output, it follows the file -name. -</P> -<P> -<b>-h</b>, <b>--no-filename</b> -Suppress the output filenames when searching multiple files. By default, -filenames are shown when multiple files are searched. For matching lines, the -filename is followed by a colon; for context lines, a hyphen separator is used. -If a line number is also being output, it follows the file name. -</P> -<P> -<b>--help</b> -Output a help message, giving brief details of the command options and file -type support, and then exit. Anything else on the command line is -ignored. -</P> -<P> -<b>-I</b> -Treat binary files as never matching. This is equivalent to -<b>--binary-files</b>=<i>without-match</i>. -</P> -<P> -<b>-i</b>, <b>--ignore-case</b> -Ignore upper/lower case distinctions during comparisons. -</P> -<P> -<b>--include</b>=<i>pattern</i> -If any <b>--include</b> patterns are specified, the only files that are -processed are those that match one of the patterns (and do not match an -<b>--exclude</b> pattern). This option does not affect directories, but it -applies to all files, whether listed on the command line, obtained from -<b>--file-list</b>, or by scanning a directory. The pattern is a PCRE regular -expression, and is matched against the final component of the file name, not -the entire path. The <b>-F</b>, <b>-w</b>, and <b>-x</b> options do not apply to -this pattern. The option may be given any number of times. If a file name -matches both an <b>--include</b> and an <b>--exclude</b> pattern, it is excluded. -There is no short form for this option. -</P> -<P> -<b>--include-from=</b><i>filename</i> -Treat each non-empty line of the file as the data for an <b>--include</b> -option. What constitutes a newline for this purpose is the operating system's -default. The <b>--newline</b> option has no effect on this option. This option -may be given any number of times; all the files are read. -</P> -<P> -<b>--include-dir</b>=<i>pattern</i> -If any <b>--include-dir</b> patterns are specified, the only directories that -are processed are those that match one of the patterns (and do not match an -<b>--exclude-dir</b> pattern). This applies to all directories, whether listed -on the command line, obtained from <b>--file-list</b>, or by scanning a parent -directory. The pattern is a PCRE regular expression, and is matched against the -final component of the directory name, not the entire path. The <b>-F</b>, -<b>-w</b>, and <b>-x</b> options do not apply to this pattern. The option may be -given any number of times. If a directory matches both <b>--include-dir</b> and -<b>--exclude-dir</b>, it is excluded. There is no short form for this option. -</P> -<P> -<b>-L</b>, <b>--files-without-match</b> -Instead of outputting lines from the files, just output the names of the files -that do not contain any lines that would have been output. Each file name is -output once, on a separate line. -</P> -<P> -<b>-l</b>, <b>--files-with-matches</b> -Instead of outputting lines from the files, just output the names of the files -containing lines that would have been output. Each file name is output -once, on a separate line. Searching normally stops as soon as a matching line -is found in a file. However, if the <b>-c</b> (count) option is also used, -matching continues in order to obtain the correct count, and those files that -have at least one match are listed along with their counts. Using this option -with <b>-c</b> is a way of suppressing the listing of files with no matches. -</P> -<P> -<b>--label</b>=<i>name</i> -This option supplies a name to be used for the standard input when file names -are being output. If not supplied, "(standard input)" is used. There is no -short form for this option. -</P> -<P> -<b>--line-buffered</b> -When this option is given, input is read and processed line by line, and the -output is flushed after each write. By default, input is read in large chunks, -unless <b>pcregrep</b> can determine that it is reading from a terminal (which -is currently possible only in Unix-like environments). Output to terminal is -normally automatically flushed by the operating system. This option can be -useful when the input or output is attached to a pipe and you do not want -<b>pcregrep</b> to buffer up large amounts of data. However, its use will affect -performance, and the <b>-M</b> (multiline) option ceases to work. -</P> -<P> -<b>--line-offsets</b> -Instead of showing lines or parts of lines that match, show each match as a -line number, the offset from the start of the line, and a length. The line -number is terminated by a colon (as usual; see the <b>-n</b> option), and the -offset and length are separated by a comma. In this mode, no context is shown. -That is, the <b>-A</b>, <b>-B</b>, and <b>-C</b> options are ignored. If there is -more than one match in a line, each of them is shown separately. This option is -mutually exclusive with <b>--file-offsets</b> and <b>--only-matching</b>. -</P> -<P> -<b>--locale</b>=<i>locale-name</i> -This option specifies a locale to be used for pattern matching. It overrides -the value in the <b>LC_ALL</b> or <b>LC_CTYPE</b> environment variables. If no -locale is specified, the PCRE library's default (usually the "C" locale) is -used. There is no short form for this option. -</P> -<P> -<b>--match-limit</b>=<i>number</i> -Processing some regular expression patterns can require a very large amount of -memory, leading in some cases to a program crash if not enough is available. -Other patterns may take a very long time to search for all possible matching -strings. The <b>pcre_exec()</b> function that is called by <b>pcregrep</b> to do -the matching has two parameters that can limit the resources that it uses. -<br> -<br> -The <b>--match-limit</b> option provides a means of limiting resource usage -when processing patterns that are not going to match, but which have a very -large number of possibilities in their search trees. The classic example is a -pattern that uses nested unlimited repeats. Internally, PCRE uses a function -called <b>match()</b> which it calls repeatedly (sometimes recursively). The -limit set by <b>--match-limit</b> is imposed on the number of times this -function is called during a match, which has the effect of limiting the amount -of backtracking that can take place. -<br> -<br> -The <b>--recursion-limit</b> option is similar to <b>--match-limit</b>, but -instead of limiting the total number of times that <b>match()</b> is called, it -limits the depth of recursive calls, which in turn limits the amount of memory -that can be used. The recursion depth is a smaller number than the total number -of calls, because not all calls to <b>match()</b> are recursive. This limit is -of use only if it is set smaller than <b>--match-limit</b>. -<br> -<br> -There are no short forms for these options. The default settings are specified -when the PCRE library is compiled, with the default default being 10 million. -</P> -<P> -<b>-M</b>, <b>--multiline</b> -Allow patterns to match more than one line. When this option is given, patterns -may usefully contain literal newline characters and internal occurrences of ^ -and $ characters. The output for a successful match may consist of more than -one line, the last of which is the one in which the match ended. If the matched -string ends with a newline sequence the output ends at the end of that line. -<br> -<br> -When this option is set, the PCRE library is called in "multiline" mode. -There is a limit to the number of lines that can be matched, imposed by the way -that <b>pcregrep</b> buffers the input file as it scans it. However, -<b>pcregrep</b> ensures that at least 8K characters or the rest of the document -(whichever is the shorter) are available for forward matching, and similarly -the previous 8K characters (or all the previous characters, if fewer than 8K) -are guaranteed to be available for lookbehind assertions. This option does not -work when input is read line by line (see \fP--line-buffered\fP.) -</P> -<P> -<b>-N</b> <i>newline-type</i>, <b>--newline</b>=<i>newline-type</i> -The PCRE library supports five different conventions for indicating -the ends of lines. They are the single-character sequences CR (carriage return) -and LF (linefeed), the two-character sequence CRLF, an "anycrlf" convention, -which recognizes any of the preceding three types, and an "any" convention, in -which any Unicode line ending sequence is assumed to end a line. The Unicode -sequences are the three just mentioned, plus VT (vertical tab, U+000B), FF -(form feed, U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and -PS (paragraph separator, U+2029). -<br> -<br> -When the PCRE library is built, a default line-ending sequence is specified. -This is normally the standard sequence for the operating system. Unless -otherwise specified by this option, <b>pcregrep</b> uses the library's default. -The possible values for this option are CR, LF, CRLF, ANYCRLF, or ANY. This -makes it possible to use <b>pcregrep</b> to scan files that have come from other -environments without having to modify their line endings. If the data that is -being scanned does not agree with the convention set by this option, -<b>pcregrep</b> may behave in strange ways. Note that this option does not -apply to files specified by the <b>-f</b>, <b>--exclude-from</b>, or -<b>--include-from</b> options, which are expected to use the operating system's -standard newline sequence. -</P> -<P> -<b>-n</b>, <b>--line-number</b> -Precede each output line by its line number in the file, followed by a colon -for matching lines or a hyphen for context lines. If the filename is also being -output, it precedes the line number. This option is forced if -<b>--line-offsets</b> is used. -</P> -<P> -<b>--no-jit</b> -If the PCRE library is built with support for just-in-time compiling (which -speeds up matching), <b>pcregrep</b> automatically makes use of this, unless it -was explicitly disabled at build time. This option can be used to disable the -use of JIT at run time. It is provided for testing and working round problems. -It should never be needed in normal use. -</P> -<P> -<b>-o</b>, <b>--only-matching</b> -Show only the part of the line that matched a pattern instead of the whole -line. In this mode, no context is shown. That is, the <b>-A</b>, <b>-B</b>, and -<b>-C</b> options are ignored. If there is more than one match in a line, each -of them is shown separately. If <b>-o</b> is combined with <b>-v</b> (invert the -sense of the match to find non-matching lines), no output is generated, but the -return code is set appropriately. If the matched portion of the line is empty, -nothing is output unless the file name or line number are being printed, in -which case they are shown on an otherwise empty line. This option is mutually -exclusive with <b>--file-offsets</b> and <b>--line-offsets</b>. -</P> -<P> -<b>-o</b><i>number</i>, <b>--only-matching</b>=<i>number</i> -Show only the part of the line that matched the capturing parentheses of the -given number. Up to 32 capturing parentheses are supported, and -o0 is -equivalent to <b>-o</b> without a number. Because these options can be given -without an argument (see above), if an argument is present, it must be given in -the same shell item, for example, -o3 or --only-matching=2. The comments given -for the non-argument case above also apply to this case. If the specified -capturing parentheses do not exist in the pattern, or were not set in the -match, nothing is output unless the file name or line number are being printed. -<br> -<br> -If this option is given multiple times, multiple substrings are output, in the -order the options are given. For example, -o3 -o1 -o3 causes the substrings -matched by capturing parentheses 3 and 1 and then 3 again to be output. By -default, there is no separator (but see the next option). -</P> -<P> -<b>--om-separator</b>=<i>text</i> -Specify a separating string for multiple occurrences of <b>-o</b>. The default -is an empty string. Separating strings are never coloured. -</P> -<P> -<b>-q</b>, <b>--quiet</b> -Work quietly, that is, display nothing except error messages. The exit -status indicates whether or not any matches were found. -</P> -<P> -<b>-r</b>, <b>--recursive</b> -If any given path is a directory, recursively scan the files it contains, -taking note of any <b>--include</b> and <b>--exclude</b> settings. By default, a -directory is read as a normal file; in some operating systems this gives an -immediate end-of-file. This option is a shorthand for setting the <b>-d</b> -option to "recurse". -</P> -<P> -<b>--recursion-limit</b>=<i>number</i> -See <b>--match-limit</b> above. -</P> -<P> -<b>-s</b>, <b>--no-messages</b> -Suppress error messages about non-existent or unreadable files. Such files are -quietly skipped. However, the return code is still 2, even if matches were -found in other files. -</P> -<P> -<b>-u</b>, <b>--utf-8</b> -Operate in UTF-8 mode. This option is available only if PCRE has been compiled -with UTF-8 support. All patterns (including those for any <b>--exclude</b> and -<b>--include</b> options) and all subject lines that are scanned must be valid -strings of UTF-8 characters. -</P> -<P> -<b>-V</b>, <b>--version</b> -Write the version numbers of <b>pcregrep</b> and the PCRE library to the -standard output and then exit. Anything else on the command line is -ignored. -</P> -<P> -<b>-v</b>, <b>--invert-match</b> -Invert the sense of the match, so that lines which do <i>not</i> match any of -the patterns are the ones that are found. -</P> -<P> -<b>-w</b>, <b>--word-regex</b>, <b>--word-regexp</b> -Force the patterns to match only whole words. This is equivalent to having \b -at the start and end of the pattern. This option applies only to the patterns -that are matched against the contents of files; it does not apply to patterns -specified by any of the <b>--include</b> or <b>--exclude</b> options. -</P> -<P> -<b>-x</b>, <b>--line-regex</b>, <b>--line-regexp</b> -Force the patterns to be anchored (each must start matching at the beginning of -a line) and in addition, require them to match entire lines. This is equivalent -to having ^ and $ characters at the start and end of each alternative branch in -every pattern. This option applies only to the patterns that are matched -against the contents of files; it does not apply to patterns specified by any -of the <b>--include</b> or <b>--exclude</b> options. -</P> -<br><a name="SEC6" href="#TOC1">ENVIRONMENT VARIABLES</a><br> -<P> -The environment variables <b>LC_ALL</b> and <b>LC_CTYPE</b> are examined, in that -order, for a locale. The first one that is set is used. This can be overridden -by the <b>--locale</b> option. If no locale is set, the PCRE library's default -(usually the "C" locale) is used. -</P> -<br><a name="SEC7" href="#TOC1">NEWLINES</a><br> -<P> -The <b>-N</b> (<b>--newline</b>) option allows <b>pcregrep</b> to scan files with -different newline conventions from the default. Any parts of the input files -that are written to the standard output are copied identically, with whatever -newline sequences they have in the input. However, the setting of this option -does not affect the interpretation of files specified by the <b>-f</b>, -<b>--exclude-from</b>, or <b>--include-from</b> options, which are assumed to use -the operating system's standard newline sequence, nor does it affect the way in -which <b>pcregrep</b> writes informational messages to the standard error and -output streams. For these it uses the string "\n" to indicate newlines, -relying on the C I/O library to convert this to an appropriate sequence. -</P> -<br><a name="SEC8" href="#TOC1">OPTIONS COMPATIBILITY</a><br> -<P> -Many of the short and long forms of <b>pcregrep</b>'s options are the same -as in the GNU <b>grep</b> program. Any long option of the form -<b>--xxx-regexp</b> (GNU terminology) is also available as <b>--xxx-regex</b> -(PCRE terminology). However, the <b>--file-list</b>, <b>--file-offsets</b>, -<b>--include-dir</b>, <b>--line-offsets</b>, <b>--locale</b>, <b>--match-limit</b>, -<b>-M</b>, <b>--multiline</b>, <b>-N</b>, <b>--newline</b>, <b>--om-separator</b>, -<b>--recursion-limit</b>, <b>-u</b>, and <b>--utf-8</b> options are specific to -<b>pcregrep</b>, as is the use of the <b>--only-matching</b> option with a -capturing parentheses number. -</P> -<P> -Although most of the common options work the same way, a few are different in -<b>pcregrep</b>. For example, the <b>--include</b> option's argument is a glob -for GNU <b>grep</b>, but a regular expression for <b>pcregrep</b>. If both the -<b>-c</b> and <b>-l</b> options are given, GNU grep lists only file names, -without counts, but <b>pcregrep</b> gives the counts. -</P> -<br><a name="SEC9" href="#TOC1">OPTIONS WITH DATA</a><br> -<P> -There are four different ways in which an option with data can be specified. -If a short form option is used, the data may follow immediately, or (with one -exception) in the next command line item. For example: -<pre> - -f/some/file - -f /some/file -</pre> -The exception is the <b>-o</b> option, which may appear with or without data. -Because of this, if data is present, it must follow immediately in the same -item, for example -o3. -</P> -<P> -If a long form option is used, the data may appear in the same command line -item, separated by an equals character, or (with two exceptions) it may appear -in the next command line item. For example: -<pre> - --file=/some/file - --file /some/file -</pre> -Note, however, that if you want to supply a file name beginning with ~ as data -in a shell command, and have the shell expand ~ to a home directory, you must -separate the file name from the option, because the shell does not treat ~ -specially unless it is at the start of an item. -</P> -<P> -The exceptions to the above are the <b>--colour</b> (or <b>--color</b>) and -<b>--only-matching</b> options, for which the data is optional. If one of these -options does have data, it must be given in the first form, using an equals -character. Otherwise <b>pcregrep</b> will assume that it has no data. -</P> -<br><a name="SEC10" href="#TOC1">MATCHING ERRORS</a><br> -<P> -It is possible to supply a regular expression that takes a very long time to -fail to match certain lines. Such patterns normally involve nested indefinite -repeats, for example: (a+)*\d when matched against a line of a's with no final -digit. The PCRE matching function has a resource limit that causes it to abort -in these circumstances. If this happens, <b>pcregrep</b> outputs an error -message and the line that caused the problem to the standard error stream. If -there are more than 20 such errors, <b>pcregrep</b> gives up. -</P> -<P> -The <b>--match-limit</b> option of <b>pcregrep</b> can be used to set the overall -resource limit; there is a second option called <b>--recursion-limit</b> that -sets a limit on the amount of memory (usually stack) that is used (see the -discussion of these options above). -</P> -<br><a name="SEC11" href="#TOC1">DIAGNOSTICS</a><br> -<P> -Exit status is 0 if any matches were found, 1 if no matches were found, and 2 -for syntax errors, overlong lines, non-existent or inaccessible files (even if -matches were found in other files) or too many matching errors. Using the -<b>-s</b> option to suppress error messages about inaccessible files does not -affect the return code. -</P> -<br><a name="SEC12" href="#TOC1">SEE ALSO</a><br> -<P> -<b>pcrepattern</b>(3), <b>pcresyntax</b>(3), <b>pcretest</b>(1). -</P> -<br><a name="SEC13" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC14" href="#TOC1">REVISION</a><br> -<P> -Last updated: 03 April 2014 -<br> -Copyright © 1997-2014 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcrejit.html b/plugins/Pcre16/docs/doc/html/pcrejit.html deleted file mode 100644 index 210f1da026..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcrejit.html +++ /dev/null @@ -1,452 +0,0 @@ -<html> -<head> -<title>pcrejit specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcrejit man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">PCRE JUST-IN-TIME COMPILER SUPPORT</a> -<li><a name="TOC2" href="#SEC2">8-BIT, 16-BIT AND 32-BIT SUPPORT</a> -<li><a name="TOC3" href="#SEC3">AVAILABILITY OF JIT SUPPORT</a> -<li><a name="TOC4" href="#SEC4">SIMPLE USE OF JIT</a> -<li><a name="TOC5" href="#SEC5">UNSUPPORTED OPTIONS AND PATTERN ITEMS</a> -<li><a name="TOC6" href="#SEC6">RETURN VALUES FROM JIT EXECUTION</a> -<li><a name="TOC7" href="#SEC7">SAVING AND RESTORING COMPILED PATTERNS</a> -<li><a name="TOC8" href="#SEC8">CONTROLLING THE JIT STACK</a> -<li><a name="TOC9" href="#SEC9">JIT STACK FAQ</a> -<li><a name="TOC10" href="#SEC10">EXAMPLE CODE</a> -<li><a name="TOC11" href="#SEC11">JIT FAST PATH API</a> -<li><a name="TOC12" href="#SEC12">SEE ALSO</a> -<li><a name="TOC13" href="#SEC13">AUTHOR</a> -<li><a name="TOC14" href="#SEC14">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">PCRE JUST-IN-TIME COMPILER SUPPORT</a><br> -<P> -Just-in-time compiling is a heavyweight optimization that can greatly speed up -pattern matching. However, it comes at the cost of extra processing before the -match is performed. Therefore, it is of most benefit when the same pattern is -going to be matched many times. This does not necessarily mean many calls of a -matching function; if the pattern is not anchored, matching attempts may take -place many times at various positions in the subject, even for a single call. -Therefore, if the subject string is very long, it may still pay to use JIT for -one-off matches. -</P> -<P> -JIT support applies only to the traditional Perl-compatible matching function. -It does not apply when the DFA matching function is being used. The code for -this support was written by Zoltan Herczeg. -</P> -<br><a name="SEC2" href="#TOC1">8-BIT, 16-BIT AND 32-BIT SUPPORT</a><br> -<P> -JIT support is available for all of the 8-bit, 16-bit and 32-bit PCRE -libraries. To keep this documentation simple, only the 8-bit interface is -described in what follows. If you are using the 16-bit library, substitute the -16-bit functions and 16-bit structures (for example, <i>pcre16_jit_stack</i> -instead of <i>pcre_jit_stack</i>). If you are using the 32-bit library, -substitute the 32-bit functions and 32-bit structures (for example, -<i>pcre32_jit_stack</i> instead of <i>pcre_jit_stack</i>). -</P> -<br><a name="SEC3" href="#TOC1">AVAILABILITY OF JIT SUPPORT</a><br> -<P> -JIT support is an optional feature of PCRE. The "configure" option --enable-jit -(or equivalent CMake option) must be set when PCRE is built if you want to use -JIT. The support is limited to the following hardware platforms: -<pre> - ARM v5, v7, and Thumb2 - Intel x86 32-bit and 64-bit - MIPS 32-bit - Power PC 32-bit and 64-bit - SPARC 32-bit (experimental) -</pre> -If --enable-jit is set on an unsupported platform, compilation fails. -</P> -<P> -A program that is linked with PCRE 8.20 or later can tell if JIT support is -available by calling <b>pcre_config()</b> with the PCRE_CONFIG_JIT option. The -result is 1 when JIT is available, and 0 otherwise. However, a simple program -does not need to check this in order to use JIT. The normal API is implemented -in a way that falls back to the interpretive code if JIT is not available. For -programs that need the best possible performance, there is also a "fast path" -API that is JIT-specific. -</P> -<P> -If your program may sometimes be linked with versions of PCRE that are older -than 8.20, but you want to use JIT when it is available, you can test -the values of PCRE_MAJOR and PCRE_MINOR, or the existence of a JIT macro such -as PCRE_CONFIG_JIT, for compile-time control of your code. -</P> -<br><a name="SEC4" href="#TOC1">SIMPLE USE OF JIT</a><br> -<P> -You have to do two things to make use of the JIT support in the simplest way: -<pre> - (1) Call <b>pcre_study()</b> with the PCRE_STUDY_JIT_COMPILE option for - each compiled pattern, and pass the resulting <b>pcre_extra</b> block to - <b>pcre_exec()</b>. - - (2) Use <b>pcre_free_study()</b> to free the <b>pcre_extra</b> block when it is - no longer needed, instead of just freeing it yourself. This ensures that - any JIT data is also freed. -</pre> -For a program that may be linked with pre-8.20 versions of PCRE, you can insert -<pre> - #ifndef PCRE_STUDY_JIT_COMPILE - #define PCRE_STUDY_JIT_COMPILE 0 - #endif -</pre> -so that no option is passed to <b>pcre_study()</b>, and then use something like -this to free the study data: -<pre> - #ifdef PCRE_CONFIG_JIT - pcre_free_study(study_ptr); - #else - pcre_free(study_ptr); - #endif -</pre> -PCRE_STUDY_JIT_COMPILE requests the JIT compiler to generate code for complete -matches. If you want to run partial matches using the PCRE_PARTIAL_HARD or -PCRE_PARTIAL_SOFT options of <b>pcre_exec()</b>, you should set one or both of -the following options in addition to, or instead of, PCRE_STUDY_JIT_COMPILE -when you call <b>pcre_study()</b>: -<pre> - PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE - PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE -</pre> -The JIT compiler generates different optimized code for each of the three -modes (normal, soft partial, hard partial). When <b>pcre_exec()</b> is called, -the appropriate code is run if it is available. Otherwise, the pattern is -matched using interpretive code. -</P> -<P> -In some circumstances you may need to call additional functions. These are -described in the section entitled -<a href="#stackcontrol">"Controlling the JIT stack"</a> -below. -</P> -<P> -If JIT support is not available, PCRE_STUDY_JIT_COMPILE etc. are ignored, and -no JIT data is created. Otherwise, the compiled pattern is passed to the JIT -compiler, which turns it into machine code that executes much faster than the -normal interpretive code. When <b>pcre_exec()</b> is passed a <b>pcre_extra</b> -block containing a pointer to JIT code of the appropriate mode (normal or -hard/soft partial), it obeys that code instead of running the interpreter. The -result is identical, but the compiled JIT code runs much faster. -</P> -<P> -There are some <b>pcre_exec()</b> options that are not supported for JIT -execution. There are also some pattern items that JIT cannot handle. Details -are given below. In both cases, execution automatically falls back to the -interpretive code. If you want to know whether JIT was actually used for a -particular match, you should arrange for a JIT callback function to be set up -as described in the section entitled -<a href="#stackcontrol">"Controlling the JIT stack"</a> -below, even if you do not need to supply a non-default JIT stack. Such a -callback function is called whenever JIT code is about to be obeyed. If the -execution options are not right for JIT execution, the callback function is not -obeyed. -</P> -<P> -If the JIT compiler finds an unsupported item, no JIT data is generated. You -can find out if JIT execution is available after studying a pattern by calling -<b>pcre_fullinfo()</b> with the PCRE_INFO_JIT option. A result of 1 means that -JIT compilation was successful. A result of 0 means that JIT support is not -available, or the pattern was not studied with PCRE_STUDY_JIT_COMPILE etc., or -the JIT compiler was not able to handle the pattern. -</P> -<P> -Once a pattern has been studied, with or without JIT, it can be used as many -times as you like for matching different subject strings. -</P> -<br><a name="SEC5" href="#TOC1">UNSUPPORTED OPTIONS AND PATTERN ITEMS</a><br> -<P> -The only <b>pcre_exec()</b> options that are supported for JIT execution are -PCRE_NO_UTF8_CHECK, PCRE_NO_UTF16_CHECK, PCRE_NO_UTF32_CHECK, PCRE_NOTBOL, -PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, PCRE_PARTIAL_HARD, and -PCRE_PARTIAL_SOFT. -</P> -<P> -The only unsupported pattern items are \C (match a single data unit) when -running in a UTF mode, and a callout immediately before an assertion condition -in a conditional group. -</P> -<br><a name="SEC6" href="#TOC1">RETURN VALUES FROM JIT EXECUTION</a><br> -<P> -When a pattern is matched using JIT execution, the return values are the same -as those given by the interpretive <b>pcre_exec()</b> code, with the addition of -one new error code: PCRE_ERROR_JIT_STACKLIMIT. This means that the memory used -for the JIT stack was insufficient. See -<a href="#stackcontrol">"Controlling the JIT stack"</a> -below for a discussion of JIT stack usage. For compatibility with the -interpretive <b>pcre_exec()</b> code, no more than two-thirds of the -<i>ovector</i> argument is used for passing back captured substrings. -</P> -<P> -The error code PCRE_ERROR_MATCHLIMIT is returned by the JIT code if searching a -very large pattern tree goes on for too long, as it is in the same circumstance -when JIT is not used, but the details of exactly what is counted are not the -same. The PCRE_ERROR_RECURSIONLIMIT error code is never returned by JIT -execution. -</P> -<br><a name="SEC7" href="#TOC1">SAVING AND RESTORING COMPILED PATTERNS</a><br> -<P> -The code that is generated by the JIT compiler is architecture-specific, and is -also position dependent. For those reasons it cannot be saved (in a file or -database) and restored later like the bytecode and other data of a compiled -pattern. Saving and restoring compiled patterns is not something many people -do. More detail about this facility is given in the -<a href="pcreprecompile.html"><b>pcreprecompile</b></a> -documentation. It should be possible to run <b>pcre_study()</b> on a saved and -restored pattern, and thereby recreate the JIT data, but because JIT -compilation uses significant resources, it is probably not worth doing this; -you might as well recompile the original pattern. -<a name="stackcontrol"></a></P> -<br><a name="SEC8" href="#TOC1">CONTROLLING THE JIT STACK</a><br> -<P> -When the compiled JIT code runs, it needs a block of memory to use as a stack. -By default, it uses 32K on the machine stack. However, some large or -complicated patterns need more than this. The error PCRE_ERROR_JIT_STACKLIMIT -is given when there is not enough stack. Three functions are provided for -managing blocks of memory for use as JIT stacks. There is further discussion -about the use of JIT stacks in the section entitled -<a href="#stackcontrol">"JIT stack FAQ"</a> -below. -</P> -<P> -The <b>pcre_jit_stack_alloc()</b> function creates a JIT stack. Its arguments -are a starting size and a maximum size, and it returns a pointer to an opaque -structure of type <b>pcre_jit_stack</b>, or NULL if there is an error. The -<b>pcre_jit_stack_free()</b> function can be used to free a stack that is no -longer needed. (For the technically minded: the address space is allocated by -mmap or VirtualAlloc.) -</P> -<P> -JIT uses far less memory for recursion than the interpretive code, -and a maximum stack size of 512K to 1M should be more than enough for any -pattern. -</P> -<P> -The <b>pcre_assign_jit_stack()</b> function specifies which stack JIT code -should use. Its arguments are as follows: -<pre> - pcre_extra *extra - pcre_jit_callback callback - void *data -</pre> -The <i>extra</i> argument must be the result of studying a pattern with -PCRE_STUDY_JIT_COMPILE etc. There are three cases for the values of the other -two options: -<pre> - (1) If <i>callback</i> is NULL and <i>data</i> is NULL, an internal 32K block - on the machine stack is used. - - (2) If <i>callback</i> is NULL and <i>data</i> is not NULL, <i>data</i> must be - a valid JIT stack, the result of calling <b>pcre_jit_stack_alloc()</b>. - - (3) If <i>callback</i> is not NULL, it must point to a function that is - called with <i>data</i> as an argument at the start of matching, in - order to set up a JIT stack. If the return from the callback - function is NULL, the internal 32K stack is used; otherwise the - return value must be a valid JIT stack, the result of calling - <b>pcre_jit_stack_alloc()</b>. -</pre> -A callback function is obeyed whenever JIT code is about to be run; it is not -obeyed when <b>pcre_exec()</b> is called with options that are incompatible for -JIT execution. A callback function can therefore be used to determine whether a -match operation was executed by JIT or by the interpreter. -</P> -<P> -You may safely use the same JIT stack for more than one pattern (either by -assigning directly or by callback), as long as the patterns are all matched -sequentially in the same thread. In a multithread application, if you do not -specify a JIT stack, or if you assign or pass back NULL from a callback, that -is thread-safe, because each thread has its own machine stack. However, if you -assign or pass back a non-NULL JIT stack, this must be a different stack for -each thread so that the application is thread-safe. -</P> -<P> -Strictly speaking, even more is allowed. You can assign the same non-NULL stack -to any number of patterns as long as they are not used for matching by multiple -threads at the same time. For example, you can assign the same stack to all -compiled patterns, and use a global mutex in the callback to wait until the -stack is available for use. However, this is an inefficient solution, and not -recommended. -</P> -<P> -This is a suggestion for how a multithreaded program that needs to set up -non-default JIT stacks might operate: -<pre> - During thread initalization - thread_local_var = pcre_jit_stack_alloc(...) - - During thread exit - pcre_jit_stack_free(thread_local_var) - - Use a one-line callback function - return thread_local_var -</pre> -All the functions described in this section do nothing if JIT is not available, -and <b>pcre_assign_jit_stack()</b> does nothing unless the <b>extra</b> argument -is non-NULL and points to a <b>pcre_extra</b> block that is the result of a -successful study with PCRE_STUDY_JIT_COMPILE etc. -<a name="stackfaq"></a></P> -<br><a name="SEC9" href="#TOC1">JIT STACK FAQ</a><br> -<P> -(1) Why do we need JIT stacks? -<br> -<br> -PCRE (and JIT) is a recursive, depth-first engine, so it needs a stack where -the local data of the current node is pushed before checking its child nodes. -Allocating real machine stack on some platforms is difficult. For example, the -stack chain needs to be updated every time if we extend the stack on PowerPC. -Although it is possible, its updating time overhead decreases performance. So -we do the recursion in memory. -</P> -<P> -(2) Why don't we simply allocate blocks of memory with <b>malloc()</b>? -<br> -<br> -Modern operating systems have a nice feature: they can reserve an address space -instead of allocating memory. We can safely allocate memory pages inside this -address space, so the stack could grow without moving memory data (this is -important because of pointers). Thus we can allocate 1M address space, and use -only a single memory page (usually 4K) if that is enough. However, we can still -grow up to 1M anytime if needed. -</P> -<P> -(3) Who "owns" a JIT stack? -<br> -<br> -The owner of the stack is the user program, not the JIT studied pattern or -anything else. The user program must ensure that if a stack is used by -<b>pcre_exec()</b>, (that is, it is assigned to the pattern currently running), -that stack must not be used by any other threads (to avoid overwriting the same -memory area). The best practice for multithreaded programs is to allocate a -stack for each thread, and return this stack through the JIT callback function. -</P> -<P> -(4) When should a JIT stack be freed? -<br> -<br> -You can free a JIT stack at any time, as long as it will not be used by -<b>pcre_exec()</b> again. When you assign the stack to a pattern, only a pointer -is set. There is no reference counting or any other magic. You can free the -patterns and stacks in any order, anytime. Just <i>do not</i> call -<b>pcre_exec()</b> with a pattern pointing to an already freed stack, as that -will cause SEGFAULT. (Also, do not free a stack currently used by -<b>pcre_exec()</b> in another thread). You can also replace the stack for a -pattern at any time. You can even free the previous stack before assigning a -replacement. -</P> -<P> -(5) Should I allocate/free a stack every time before/after calling -<b>pcre_exec()</b>? -<br> -<br> -No, because this is too costly in terms of resources. However, you could -implement some clever idea which release the stack if it is not used in let's -say two minutes. The JIT callback can help to achieve this without keeping a -list of the currently JIT studied patterns. -</P> -<P> -(6) OK, the stack is for long term memory allocation. But what happens if a -pattern causes stack overflow with a stack of 1M? Is that 1M kept until the -stack is freed? -<br> -<br> -Especially on embedded sytems, it might be a good idea to release memory -sometimes without freeing the stack. There is no API for this at the moment. -Probably a function call which returns with the currently allocated memory for -any stack and another which allows releasing memory (shrinking the stack) would -be a good idea if someone needs this. -</P> -<P> -(7) This is too much of a headache. Isn't there any better solution for JIT -stack handling? -<br> -<br> -No, thanks to Windows. If POSIX threads were used everywhere, we could throw -out this complicated API. -</P> -<br><a name="SEC10" href="#TOC1">EXAMPLE CODE</a><br> -<P> -This is a single-threaded example that specifies a JIT stack without using a -callback. -<pre> - int rc; - int ovector[30]; - pcre *re; - pcre_extra *extra; - pcre_jit_stack *jit_stack; - - re = pcre_compile(pattern, 0, &error, &erroffset, NULL); - /* Check for errors */ - extra = pcre_study(re, PCRE_STUDY_JIT_COMPILE, &error); - jit_stack = pcre_jit_stack_alloc(32*1024, 512*1024); - /* Check for error (NULL) */ - pcre_assign_jit_stack(extra, NULL, jit_stack); - rc = pcre_exec(re, extra, subject, length, 0, 0, ovector, 30); - /* Check results */ - pcre_free(re); - pcre_free_study(extra); - pcre_jit_stack_free(jit_stack); - -</PRE> -</P> -<br><a name="SEC11" href="#TOC1">JIT FAST PATH API</a><br> -<P> -Because the API described above falls back to interpreted execution when JIT is -not available, it is convenient for programs that are written for general use -in many environments. However, calling JIT via <b>pcre_exec()</b> does have a -performance impact. Programs that are written for use where JIT is known to be -available, and which need the best possible performance, can instead use a -"fast path" API to call JIT execution directly instead of calling -<b>pcre_exec()</b> (obviously only for patterns that have been successfully -studied by JIT). -</P> -<P> -The fast path function is called <b>pcre_jit_exec()</b>, and it takes exactly -the same arguments as <b>pcre_exec()</b>, plus one additional argument that -must point to a JIT stack. The JIT stack arrangements described above do not -apply. The return values are the same as for <b>pcre_exec()</b>. -</P> -<P> -When you call <b>pcre_exec()</b>, as well as testing for invalid options, a -number of other sanity checks are performed on the arguments. For example, if -the subject pointer is NULL, or its length is negative, an immediate error is -given. Also, unless PCRE_NO_UTF[8|16|32] is set, a UTF subject string is tested -for validity. In the interests of speed, these checks do not happen on the JIT -fast path, and if invalid data is passed, the result is undefined. -</P> -<P> -Bypassing the sanity checks and the <b>pcre_exec()</b> wrapping can give -speedups of more than 10%. -</P> -<br><a name="SEC12" href="#TOC1">SEE ALSO</a><br> -<P> -<b>pcreapi</b>(3) -</P> -<br><a name="SEC13" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel (FAQ by Zoltan Herczeg) -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC14" href="#TOC1">REVISION</a><br> -<P> -Last updated: 17 March 2013 -<br> -Copyright © 1997-2013 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcrelimits.html b/plugins/Pcre16/docs/doc/html/pcrelimits.html deleted file mode 100644 index ee5ebf033d..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcrelimits.html +++ /dev/null @@ -1,90 +0,0 @@ -<html> -<head> -<title>pcrelimits specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcrelimits man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -SIZE AND OTHER LIMITATIONS -</b><br> -<P> -There are some size limitations in PCRE but it is hoped that they will never in -practice be relevant. -</P> -<P> -The maximum length of a compiled pattern is approximately 64K data units (bytes -for the 8-bit library, 16-bit units for the 16-bit library, and 32-bit units for -the 32-bit library) if PCRE is compiled with the default internal linkage size, -which is 2 bytes for the 8-bit and 16-bit libraries, and 4 bytes for the 32-bit -library. If you want to process regular expressions that are truly enormous, -you can compile PCRE with an internal linkage size of 3 or 4 (when building the -16-bit or 32-bit library, 3 is rounded up to 4). See the <b>README</b> file in -the source distribution and the -<a href="pcrebuild.html"><b>pcrebuild</b></a> -documentation for details. In these cases the limit is substantially larger. -However, the speed of execution is slower. -</P> -<P> -All values in repeating quantifiers must be less than 65536. -</P> -<P> -There is no limit to the number of parenthesized subpatterns, but there can be -no more than 65535 capturing subpatterns. There is, however, a limit to the -depth of nesting of parenthesized subpatterns of all kinds. This is imposed in -order to limit the amount of system stack used at compile time. The limit can -be specified when PCRE is built; the default is 250. -</P> -<P> -There is a limit to the number of forward references to subsequent subpatterns -of around 200,000. Repeated forward references with fixed upper limits, for -example, (?2){0,100} when subpattern number 2 is to the right, are included in -the count. There is no limit to the number of backward references. -</P> -<P> -The maximum length of name for a named subpattern is 32 characters, and the -maximum number of named subpatterns is 10000. -</P> -<P> -The maximum length of a name in a (*MARK), (*PRUNE), (*SKIP), or (*THEN) verb -is 255 for the 8-bit library and 65535 for the 16-bit and 32-bit libraries. -</P> -<P> -The maximum length of a subject string is the largest positive number that an -integer variable can hold. However, when using the traditional matching -function, PCRE uses recursion to handle subpatterns and indefinite repetition. -This means that the available stack space may limit the size of a subject -string that can be processed by certain patterns. For a discussion of stack -issues, see the -<a href="pcrestack.html"><b>pcrestack</b></a> -documentation. -</P> -<br><b> -AUTHOR -</b><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><b> -REVISION -</b><br> -<P> -Last updated: 05 November 2013 -<br> -Copyright © 1997-2013 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcrematching.html b/plugins/Pcre16/docs/doc/html/pcrematching.html deleted file mode 100644 index a1af39b68d..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcrematching.html +++ /dev/null @@ -1,242 +0,0 @@ -<html> -<head> -<title>pcrematching specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcrematching man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">PCRE MATCHING ALGORITHMS</a> -<li><a name="TOC2" href="#SEC2">REGULAR EXPRESSIONS AS TREES</a> -<li><a name="TOC3" href="#SEC3">THE STANDARD MATCHING ALGORITHM</a> -<li><a name="TOC4" href="#SEC4">THE ALTERNATIVE MATCHING ALGORITHM</a> -<li><a name="TOC5" href="#SEC5">ADVANTAGES OF THE ALTERNATIVE ALGORITHM</a> -<li><a name="TOC6" href="#SEC6">DISADVANTAGES OF THE ALTERNATIVE ALGORITHM</a> -<li><a name="TOC7" href="#SEC7">AUTHOR</a> -<li><a name="TOC8" href="#SEC8">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">PCRE MATCHING ALGORITHMS</a><br> -<P> -This document describes the two different algorithms that are available in PCRE -for matching a compiled regular expression against a given subject string. The -"standard" algorithm is the one provided by the <b>pcre_exec()</b>, -<b>pcre16_exec()</b> and <b>pcre32_exec()</b> functions. These work in the same -as as Perl's matching function, and provide a Perl-compatible matching operation. -The just-in-time (JIT) optimization that is described in the -<a href="pcrejit.html"><b>pcrejit</b></a> -documentation is compatible with these functions. -</P> -<P> -An alternative algorithm is provided by the <b>pcre_dfa_exec()</b>, -<b>pcre16_dfa_exec()</b> and <b>pcre32_dfa_exec()</b> functions; they operate in -a different way, and are not Perl-compatible. This alternative has advantages -and disadvantages compared with the standard algorithm, and these are described -below. -</P> -<P> -When there is only one possible way in which a given subject string can match a -pattern, the two algorithms give the same answer. A difference arises, however, -when there are multiple possibilities. For example, if the pattern -<pre> - ^<.*> -</pre> -is matched against the string -<pre> - <something> <something else> <something further> -</pre> -there are three possible answers. The standard algorithm finds only one of -them, whereas the alternative algorithm finds all three. -</P> -<br><a name="SEC2" href="#TOC1">REGULAR EXPRESSIONS AS TREES</a><br> -<P> -The set of strings that are matched by a regular expression can be represented -as a tree structure. An unlimited repetition in the pattern makes the tree of -infinite size, but it is still a tree. Matching the pattern to a given subject -string (from a given starting point) can be thought of as a search of the tree. -There are two ways to search a tree: depth-first and breadth-first, and these -correspond to the two matching algorithms provided by PCRE. -</P> -<br><a name="SEC3" href="#TOC1">THE STANDARD MATCHING ALGORITHM</a><br> -<P> -In the terminology of Jeffrey Friedl's book "Mastering Regular -Expressions", the standard algorithm is an "NFA algorithm". It conducts a -depth-first search of the pattern tree. That is, it proceeds along a single -path through the tree, checking that the subject matches what is required. When -there is a mismatch, the algorithm tries any alternatives at the current point, -and if they all fail, it backs up to the previous branch point in the tree, and -tries the next alternative branch at that level. This often involves backing up -(moving to the left) in the subject string as well. The order in which -repetition branches are tried is controlled by the greedy or ungreedy nature of -the quantifier. -</P> -<P> -If a leaf node is reached, a matching string has been found, and at that point -the algorithm stops. Thus, if there is more than one possible match, this -algorithm returns the first one that it finds. Whether this is the shortest, -the longest, or some intermediate length depends on the way the greedy and -ungreedy repetition quantifiers are specified in the pattern. -</P> -<P> -Because it ends up with a single path through the tree, it is relatively -straightforward for this algorithm to keep track of the substrings that are -matched by portions of the pattern in parentheses. This provides support for -capturing parentheses and back references. -</P> -<br><a name="SEC4" href="#TOC1">THE ALTERNATIVE MATCHING ALGORITHM</a><br> -<P> -This algorithm conducts a breadth-first search of the tree. Starting from the -first matching point in the subject, it scans the subject string from left to -right, once, character by character, and as it does this, it remembers all the -paths through the tree that represent valid matches. In Friedl's terminology, -this is a kind of "DFA algorithm", though it is not implemented as a -traditional finite state machine (it keeps multiple states active -simultaneously). -</P> -<P> -Although the general principle of this matching algorithm is that it scans the -subject string only once, without backtracking, there is one exception: when a -lookaround assertion is encountered, the characters following or preceding the -current point have to be independently inspected. -</P> -<P> -The scan continues until either the end of the subject is reached, or there are -no more unterminated paths. At this point, terminated paths represent the -different matching possibilities (if there are none, the match has failed). -Thus, if there is more than one possible match, this algorithm finds all of -them, and in particular, it finds the longest. The matches are returned in -decreasing order of length. There is an option to stop the algorithm after the -first match (which is necessarily the shortest) is found. -</P> -<P> -Note that all the matches that are found start at the same point in the -subject. If the pattern -<pre> - cat(er(pillar)?)? -</pre> -is matched against the string "the caterpillar catchment", the result will be -the three strings "caterpillar", "cater", and "cat" that start at the fifth -character of the subject. The algorithm does not automatically move on to find -matches that start at later positions. -</P> -<P> -PCRE's "auto-possessification" optimization usually applies to character -repeats at the end of a pattern (as well as internally). For example, the -pattern "a\d+" is compiled as if it were "a\d++" because there is no point -even considering the possibility of backtracking into the repeated digits. For -DFA matching, this means that only one possible match is found. If you really -do want multiple matches in such cases, either use an ungreedy repeat -("a\d+?") or set the PCRE_NO_AUTO_POSSESS option when compiling. -</P> -<P> -There are a number of features of PCRE regular expressions that are not -supported by the alternative matching algorithm. They are as follows: -</P> -<P> -1. Because the algorithm finds all possible matches, the greedy or ungreedy -nature of repetition quantifiers is not relevant. Greedy and ungreedy -quantifiers are treated in exactly the same way. However, possessive -quantifiers can make a difference when what follows could also match what is -quantified, for example in a pattern like this: -<pre> - ^a++\w! -</pre> -This pattern matches "aaab!" but not "aaa!", which would be matched by a -non-possessive quantifier. Similarly, if an atomic group is present, it is -matched as if it were a standalone pattern at the current point, and the -longest match is then "locked in" for the rest of the overall pattern. -</P> -<P> -2. When dealing with multiple paths through the tree simultaneously, it is not -straightforward to keep track of captured substrings for the different matching -possibilities, and PCRE's implementation of this algorithm does not attempt to -do this. This means that no captured substrings are available. -</P> -<P> -3. Because no substrings are captured, back references within the pattern are -not supported, and cause errors if encountered. -</P> -<P> -4. For the same reason, conditional expressions that use a backreference as the -condition or test for a specific group recursion are not supported. -</P> -<P> -5. Because many paths through the tree may be active, the \K escape sequence, -which resets the start of the match when encountered (but may be on some paths -and not on others), is not supported. It causes an error if encountered. -</P> -<P> -6. Callouts are supported, but the value of the <i>capture_top</i> field is -always 1, and the value of the <i>capture_last</i> field is always -1. -</P> -<P> -7. The \C escape sequence, which (in the standard algorithm) always matches a -single data unit, even in UTF-8, UTF-16 or UTF-32 modes, is not supported in -these modes, because the alternative algorithm moves through the subject string -one character (not data unit) at a time, for all active paths through the tree. -</P> -<P> -8. Except for (*FAIL), the backtracking control verbs such as (*PRUNE) are not -supported. (*FAIL) is supported, and behaves like a failing negative assertion. -</P> -<br><a name="SEC5" href="#TOC1">ADVANTAGES OF THE ALTERNATIVE ALGORITHM</a><br> -<P> -Using the alternative matching algorithm provides the following advantages: -</P> -<P> -1. All possible matches (at a single point in the subject) are automatically -found, and in particular, the longest match is found. To find more than one -match using the standard algorithm, you have to do kludgy things with -callouts. -</P> -<P> -2. Because the alternative algorithm scans the subject string just once, and -never needs to backtrack (except for lookbehinds), it is possible to pass very -long subject strings to the matching function in several pieces, checking for -partial matching each time. Although it is possible to do multi-segment -matching using the standard algorithm by retaining partially matched -substrings, it is more complicated. The -<a href="pcrepartial.html"><b>pcrepartial</b></a> -documentation gives details of partial matching and discusses multi-segment -matching. -</P> -<br><a name="SEC6" href="#TOC1">DISADVANTAGES OF THE ALTERNATIVE ALGORITHM</a><br> -<P> -The alternative algorithm suffers from a number of disadvantages: -</P> -<P> -1. It is substantially slower than the standard algorithm. This is partly -because it has to search for all possible matches, but is also because it is -less susceptible to optimization. -</P> -<P> -2. Capturing parentheses and back references are not supported. -</P> -<P> -3. Although atomic groups are supported, their use does not provide the -performance advantage that it does for the standard algorithm. -</P> -<br><a name="SEC7" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC8" href="#TOC1">REVISION</a><br> -<P> -Last updated: 12 November 2013 -<br> -Copyright © 1997-2012 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcrepartial.html b/plugins/Pcre16/docs/doc/html/pcrepartial.html deleted file mode 100644 index 4faeafcb68..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcrepartial.html +++ /dev/null @@ -1,509 +0,0 @@ -<html> -<head> -<title>pcrepartial specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcrepartial man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">PARTIAL MATCHING IN PCRE</a> -<li><a name="TOC2" href="#SEC2">PARTIAL MATCHING USING pcre_exec() OR pcre[16|32]_exec()</a> -<li><a name="TOC3" href="#SEC3">PARTIAL MATCHING USING pcre_dfa_exec() OR pcre[16|32]_dfa_exec()</a> -<li><a name="TOC4" href="#SEC4">PARTIAL MATCHING AND WORD BOUNDARIES</a> -<li><a name="TOC5" href="#SEC5">FORMERLY RESTRICTED PATTERNS</a> -<li><a name="TOC6" href="#SEC6">EXAMPLE OF PARTIAL MATCHING USING PCRETEST</a> -<li><a name="TOC7" href="#SEC7">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre[16|32]_dfa_exec()</a> -<li><a name="TOC8" href="#SEC8">MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre[16|32]_exec()</a> -<li><a name="TOC9" href="#SEC9">ISSUES WITH MULTI-SEGMENT MATCHING</a> -<li><a name="TOC10" href="#SEC10">AUTHOR</a> -<li><a name="TOC11" href="#SEC11">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">PARTIAL MATCHING IN PCRE</a><br> -<P> -In normal use of PCRE, if the subject string that is passed to a matching -function matches as far as it goes, but is too short to match the entire -pattern, PCRE_ERROR_NOMATCH is returned. There are circumstances where it might -be helpful to distinguish this case from other cases in which there is no -match. -</P> -<P> -Consider, for example, an application where a human is required to type in data -for a field with specific formatting requirements. An example might be a date -in the form <i>ddmmmyy</i>, defined by this pattern: -<pre> - ^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$ -</pre> -If the application sees the user's keystrokes one by one, and can check that -what has been typed so far is potentially valid, it is able to raise an error -as soon as a mistake is made, by beeping and not reflecting the character that -has been typed, for example. This immediate feedback is likely to be a better -user interface than a check that is delayed until the entire string has been -entered. Partial matching can also be useful when the subject string is very -long and is not all available at once. -</P> -<P> -PCRE supports partial matching by means of the PCRE_PARTIAL_SOFT and -PCRE_PARTIAL_HARD options, which can be set when calling any of the matching -functions. For backwards compatibility, PCRE_PARTIAL is a synonym for -PCRE_PARTIAL_SOFT. The essential difference between the two options is whether -or not a partial match is preferred to an alternative complete match, though -the details differ between the two types of matching function. If both options -are set, PCRE_PARTIAL_HARD takes precedence. -</P> -<P> -If you want to use partial matching with just-in-time optimized code, you must -call <b>pcre_study()</b>, <b>pcre16_study()</b> or <b>pcre32_study()</b> with one -or both of these options: -<pre> - PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE - PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE -</pre> -PCRE_STUDY_JIT_COMPILE should also be set if you are going to run non-partial -matches on the same pattern. If the appropriate JIT study mode has not been set -for a match, the interpretive matching code is used. -</P> -<P> -Setting a partial matching option disables two of PCRE's standard -optimizations. PCRE remembers the last literal data unit in a pattern, and -abandons matching immediately if it is not present in the subject string. This -optimization cannot be used for a subject string that might match only -partially. If the pattern was studied, PCRE knows the minimum length of a -matching string, and does not bother to run the matching function on shorter -strings. This optimization is also disabled for partial matching. -</P> -<br><a name="SEC2" href="#TOC1">PARTIAL MATCHING USING pcre_exec() OR pcre[16|32]_exec()</a><br> -<P> -A partial match occurs during a call to <b>pcre_exec()</b> or -<b>pcre[16|32]_exec()</b> when the end of the subject string is reached -successfully, but matching cannot continue because more characters are needed. -However, at least one character in the subject must have been inspected. This -character need not form part of the final matched string; lookbehind assertions -and the \K escape sequence provide ways of inspecting characters before the -start of a matched substring. The requirement for inspecting at least one -character exists because an empty string can always be matched; without such a -restriction there would always be a partial match of an empty string at the end -of the subject. -</P> -<P> -If there are at least two slots in the offsets vector when a partial match is -returned, the first slot is set to the offset of the earliest character that -was inspected. For convenience, the second offset points to the end of the -subject so that a substring can easily be identified. If there are at least -three slots in the offsets vector, the third slot is set to the offset of the -character where matching started. -</P> -<P> -For the majority of patterns, the contents of the first and third slots will be -the same. However, for patterns that contain lookbehind assertions, or begin -with \b or \B, characters before the one where matching started may have been -inspected while carrying out the match. For example, consider this pattern: -<pre> - /(?<=abc)123/ -</pre> -This pattern matches "123", but only if it is preceded by "abc". If the subject -string is "xyzabc12", the first two offsets after a partial match are for the -substring "abc12", because all these characters were inspected. However, the -third offset is set to 6, because that is the offset where matching began. -</P> -<P> -What happens when a partial match is identified depends on which of the two -partial matching options are set. -</P> -<br><b> -PCRE_PARTIAL_SOFT WITH pcre_exec() OR pcre[16|32]_exec() -</b><br> -<P> -If PCRE_PARTIAL_SOFT is set when <b>pcre_exec()</b> or <b>pcre[16|32]_exec()</b> -identifies a partial match, the partial match is remembered, but matching -continues as normal, and other alternatives in the pattern are tried. If no -complete match can be found, PCRE_ERROR_PARTIAL is returned instead of -PCRE_ERROR_NOMATCH. -</P> -<P> -This option is "soft" because it prefers a complete match over a partial match. -All the various matching items in a pattern behave as if the subject string is -potentially complete. For example, \z, \Z, and $ match at the end of the -subject, as normal, and for \b and \B the end of the subject is treated as a -non-alphanumeric. -</P> -<P> -If there is more than one partial match, the first one that was found provides -the data that is returned. Consider this pattern: -<pre> - /123\w+X|dogY/ -</pre> -If this is matched against the subject string "abc123dog", both -alternatives fail to match, but the end of the subject is reached during -matching, so PCRE_ERROR_PARTIAL is returned. The offsets are set to 3 and 9, -identifying "123dog" as the first partial match that was found. (In this -example, there are two partial matches, because "dog" on its own partially -matches the second alternative.) -</P> -<br><b> -PCRE_PARTIAL_HARD WITH pcre_exec() OR pcre[16|32]_exec() -</b><br> -<P> -If PCRE_PARTIAL_HARD is set for <b>pcre_exec()</b> or <b>pcre[16|32]_exec()</b>, -PCRE_ERROR_PARTIAL is returned as soon as a partial match is found, without -continuing to search for possible complete matches. This option is "hard" -because it prefers an earlier partial match over a later complete match. For -this reason, the assumption is made that the end of the supplied subject string -may not be the true end of the available data, and so, if \z, \Z, \b, \B, -or $ are encountered at the end of the subject, the result is -PCRE_ERROR_PARTIAL, provided that at least one character in the subject has -been inspected. -</P> -<P> -Setting PCRE_PARTIAL_HARD also affects the way UTF-8 and UTF-16 -subject strings are checked for validity. Normally, an invalid sequence -causes the error PCRE_ERROR_BADUTF8 or PCRE_ERROR_BADUTF16. However, in the -special case of a truncated character at the end of the subject, -PCRE_ERROR_SHORTUTF8 or PCRE_ERROR_SHORTUTF16 is returned when -PCRE_PARTIAL_HARD is set. -</P> -<br><b> -Comparing hard and soft partial matching -</b><br> -<P> -The difference between the two partial matching options can be illustrated by a -pattern such as: -<pre> - /dog(sbody)?/ -</pre> -This matches either "dog" or "dogsbody", greedily (that is, it prefers the -longer string if possible). If it is matched against the string "dog" with -PCRE_PARTIAL_SOFT, it yields a complete match for "dog". However, if -PCRE_PARTIAL_HARD is set, the result is PCRE_ERROR_PARTIAL. On the other hand, -if the pattern is made ungreedy the result is different: -<pre> - /dog(sbody)??/ -</pre> -In this case the result is always a complete match because that is found first, -and matching never continues after finding a complete match. It might be easier -to follow this explanation by thinking of the two patterns like this: -<pre> - /dog(sbody)?/ is the same as /dogsbody|dog/ - /dog(sbody)??/ is the same as /dog|dogsbody/ -</pre> -The second pattern will never match "dogsbody", because it will always find the -shorter match first. -</P> -<br><a name="SEC3" href="#TOC1">PARTIAL MATCHING USING pcre_dfa_exec() OR pcre[16|32]_dfa_exec()</a><br> -<P> -The DFA functions move along the subject string character by character, without -backtracking, searching for all possible matches simultaneously. If the end of -the subject is reached before the end of the pattern, there is the possibility -of a partial match, again provided that at least one character has been -inspected. -</P> -<P> -When PCRE_PARTIAL_SOFT is set, PCRE_ERROR_PARTIAL is returned only if there -have been no complete matches. Otherwise, the complete matches are returned. -However, if PCRE_PARTIAL_HARD is set, a partial match takes precedence over any -complete matches. The portion of the string that was inspected when the longest -partial match was found is set as the first matching string, provided there are -at least two slots in the offsets vector. -</P> -<P> -Because the DFA functions always search for all possible matches, and there is -no difference between greedy and ungreedy repetition, their behaviour is -different from the standard functions when PCRE_PARTIAL_HARD is set. Consider -the string "dog" matched against the ungreedy pattern shown above: -<pre> - /dog(sbody)??/ -</pre> -Whereas the standard functions stop as soon as they find the complete match for -"dog", the DFA functions also find the partial match for "dogsbody", and so -return that when PCRE_PARTIAL_HARD is set. -</P> -<br><a name="SEC4" href="#TOC1">PARTIAL MATCHING AND WORD BOUNDARIES</a><br> -<P> -If a pattern ends with one of sequences \b or \B, which test for word -boundaries, partial matching with PCRE_PARTIAL_SOFT can give counter-intuitive -results. Consider this pattern: -<pre> - /\bcat\b/ -</pre> -This matches "cat", provided there is a word boundary at either end. If the -subject string is "the cat", the comparison of the final "t" with a following -character cannot take place, so a partial match is found. However, normal -matching carries on, and \b matches at the end of the subject when the last -character is a letter, so a complete match is found. The result, therefore, is -<i>not</i> PCRE_ERROR_PARTIAL. Using PCRE_PARTIAL_HARD in this case does yield -PCRE_ERROR_PARTIAL, because then the partial match takes precedence. -</P> -<br><a name="SEC5" href="#TOC1">FORMERLY RESTRICTED PATTERNS</a><br> -<P> -For releases of PCRE prior to 8.00, because of the way certain internal -optimizations were implemented in the <b>pcre_exec()</b> function, the -PCRE_PARTIAL option (predecessor of PCRE_PARTIAL_SOFT) could not be used with -all patterns. From release 8.00 onwards, the restrictions no longer apply, and -partial matching with can be requested for any pattern. -</P> -<P> -Items that were formerly restricted were repeated single characters and -repeated metasequences. If PCRE_PARTIAL was set for a pattern that did not -conform to the restrictions, <b>pcre_exec()</b> returned the error code -PCRE_ERROR_BADPARTIAL (-13). This error code is no longer in use. The -PCRE_INFO_OKPARTIAL call to <b>pcre_fullinfo()</b> to find out if a compiled -pattern can be used for partial matching now always returns 1. -</P> -<br><a name="SEC6" href="#TOC1">EXAMPLE OF PARTIAL MATCHING USING PCRETEST</a><br> -<P> -If the escape sequence \P is present in a <b>pcretest</b> data line, the -PCRE_PARTIAL_SOFT option is used for the match. Here is a run of <b>pcretest</b> -that uses the date example quoted above: -<pre> - re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ - data> 25jun04\P - 0: 25jun04 - 1: jun - data> 25dec3\P - Partial match: 23dec3 - data> 3ju\P - Partial match: 3ju - data> 3juj\P - No match - data> j\P - No match -</pre> -The first data string is matched completely, so <b>pcretest</b> shows the -matched substrings. The remaining four strings do not match the complete -pattern, but the first two are partial matches. Similar output is obtained -if DFA matching is used. -</P> -<P> -If the escape sequence \P is present more than once in a <b>pcretest</b> data -line, the PCRE_PARTIAL_HARD option is set for the match. -</P> -<br><a name="SEC7" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre[16|32]_dfa_exec()</a><br> -<P> -When a partial match has been found using a DFA matching function, it is -possible to continue the match by providing additional subject data and calling -the function again with the same compiled regular expression, this time setting -the PCRE_DFA_RESTART option. You must pass the same working space as before, -because this is where details of the previous partial match are stored. Here is -an example using <b>pcretest</b>, using the \R escape sequence to set the -PCRE_DFA_RESTART option (\D specifies the use of the DFA matching function): -<pre> - re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ - data> 23ja\P\D - Partial match: 23ja - data> n05\R\D - 0: n05 -</pre> -The first call has "23ja" as the subject, and requests partial matching; the -second call has "n05" as the subject for the continued (restarted) match. -Notice that when the match is complete, only the last part is shown; PCRE does -not retain the previously partially-matched string. It is up to the calling -program to do that if it needs to. -</P> -<P> -That means that, for an unanchored pattern, if a continued match fails, it is -not possible to try again at a new starting point. All this facility is capable -of doing is continuing with the previous match attempt. In the previous -example, if the second set of data is "ug23" the result is no match, even -though there would be a match for "aug23" if the entire string were given at -once. Depending on the application, this may or may not be what you want. -The only way to allow for starting again at the next character is to retain the -matched part of the subject and try a new complete match. -</P> -<P> -You can set the PCRE_PARTIAL_SOFT or PCRE_PARTIAL_HARD options with -PCRE_DFA_RESTART to continue partial matching over multiple segments. This -facility can be used to pass very long subject strings to the DFA matching -functions. -</P> -<br><a name="SEC8" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre[16|32]_exec()</a><br> -<P> -From release 8.00, the standard matching functions can also be used to do -multi-segment matching. Unlike the DFA functions, it is not possible to -restart the previous match with a new segment of data. Instead, new data must -be added to the previous subject string, and the entire match re-run, starting -from the point where the partial match occurred. Earlier data can be discarded. -</P> -<P> -It is best to use PCRE_PARTIAL_HARD in this situation, because it does not -treat the end of a segment as the end of the subject when matching \z, \Z, -\b, \B, and $. Consider an unanchored pattern that matches dates: -<pre> - re> /\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d/ - data> The date is 23ja\P\P - Partial match: 23ja -</pre> -At this stage, an application could discard the text preceding "23ja", add on -text from the next segment, and call the matching function again. Unlike the -DFA matching functions, the entire matching string must always be available, -and the complete matching process occurs for each call, so more memory and more -processing time is needed. -</P> -<P> -<b>Note:</b> If the pattern contains lookbehind assertions, or \K, or starts -with \b or \B, the string that is returned for a partial match includes -characters that precede the start of what would be returned for a complete -match, because it contains all the characters that were inspected during the -partial match. -</P> -<br><a name="SEC9" href="#TOC1">ISSUES WITH MULTI-SEGMENT MATCHING</a><br> -<P> -Certain types of pattern may give problems with multi-segment matching, -whichever matching function is used. -</P> -<P> -1. If the pattern contains a test for the beginning of a line, you need to pass -the PCRE_NOTBOL option when the subject string for any call does start at the -beginning of a line. There is also a PCRE_NOTEOL option, but in practice when -doing multi-segment matching you should be using PCRE_PARTIAL_HARD, which -includes the effect of PCRE_NOTEOL. -</P> -<P> -2. Lookbehind assertions that have already been obeyed are catered for in the -offsets that are returned for a partial match. However a lookbehind assertion -later in the pattern could require even earlier characters to be inspected. You -can handle this case by using the PCRE_INFO_MAXLOOKBEHIND option of the -<b>pcre_fullinfo()</b> or <b>pcre[16|32]_fullinfo()</b> functions to obtain the -length of the longest lookbehind in the pattern. This length is given in -characters, not bytes. If you always retain at least that many characters -before the partially matched string, all should be well. (Of course, near the -start of the subject, fewer characters may be present; in that case all -characters should be retained.) -</P> -<P> -From release 8.33, there is a more accurate way of deciding which characters to -retain. Instead of subtracting the length of the longest lookbehind from the -earliest inspected character (<i>offsets[0]</i>), the match start position -(<i>offsets[2]</i>) should be used, and the next match attempt started at the -<i>offsets[2]</i> character by setting the <i>startoffset</i> argument of -<b>pcre_exec()</b> or <b>pcre_dfa_exec()</b>. -</P> -<P> -For example, if the pattern "(?<=123)abc" is partially -matched against the string "xx123a", the three offset values returned are 2, 6, -and 5. This indicates that the matching process that gave a partial match -started at offset 5, but the characters "123a" were all inspected. The maximum -lookbehind for that pattern is 3, so taking that away from 5 shows that we need -only keep "123a", and the next match attempt can be started at offset 3 (that -is, at "a") when further characters have been added. When the match start is -not the earliest inspected character, <b>pcretest</b> shows it explicitly: -<pre> - re> "(?<=123)abc" - data> xx123a\P\P - Partial match at offset 5: 123a -</PRE> -</P> -<P> -3. Because a partial match must always contain at least one character, what -might be considered a partial match of an empty string actually gives a "no -match" result. For example: -<pre> - re> /c(?<=abc)x/ - data> ab\P - No match -</pre> -If the next segment begins "cx", a match should be found, but this will only -happen if characters from the previous segment are retained. For this reason, a -"no match" result should be interpreted as "partial match of an empty string" -when the pattern contains lookbehinds. -</P> -<P> -4. Matching a subject string that is split into multiple segments may not -always produce exactly the same result as matching over one single long string, -especially when PCRE_PARTIAL_SOFT is used. The section "Partial Matching and -Word Boundaries" above describes an issue that arises if the pattern ends with -\b or \B. Another kind of difference may occur when there are multiple -matching possibilities, because (for PCRE_PARTIAL_SOFT) a partial match result -is given only when there are no completed matches. This means that as soon as -the shortest match has been found, continuation to a new subject segment is no -longer possible. Consider again this <b>pcretest</b> example: -<pre> - re> /dog(sbody)?/ - data> dogsb\P - 0: dog - data> do\P\D - Partial match: do - data> gsb\R\P\D - 0: g - data> dogsbody\D - 0: dogsbody - 1: dog -</pre> -The first data line passes the string "dogsb" to a standard matching function, -setting the PCRE_PARTIAL_SOFT option. Although the string is a partial match -for "dogsbody", the result is not PCRE_ERROR_PARTIAL, because the shorter -string "dog" is a complete match. Similarly, when the subject is presented to -a DFA matching function in several parts ("do" and "gsb" being the first two) -the match stops when "dog" has been found, and it is not possible to continue. -On the other hand, if "dogsbody" is presented as a single string, a DFA -matching function finds both matches. -</P> -<P> -Because of these problems, it is best to use PCRE_PARTIAL_HARD when matching -multi-segment data. The example above then behaves differently: -<pre> - re> /dog(sbody)?/ - data> dogsb\P\P - Partial match: dogsb - data> do\P\D - Partial match: do - data> gsb\R\P\P\D - Partial match: gsb -</pre> -5. Patterns that contain alternatives at the top level which do not all start -with the same pattern item may not work as expected when PCRE_DFA_RESTART is -used. For example, consider this pattern: -<pre> - 1234|3789 -</pre> -If the first part of the subject is "ABC123", a partial match of the first -alternative is found at offset 3. There is no partial match for the second -alternative, because such a match does not start at the same point in the -subject string. Attempting to continue with the string "7890" does not yield a -match because only those alternatives that match at one point in the subject -are remembered. The problem arises because the start of the second alternative -matches within the first alternative. There is no problem with anchored -patterns or patterns such as: -<pre> - 1234|ABCD -</pre> -where no string can be a partial match for both alternatives. This is not a -problem if a standard matching function is used, because the entire match has -to be rerun each time: -<pre> - re> /1234|3789/ - data> ABC123\P\P - Partial match: 123 - data> 1237890 - 0: 3789 -</pre> -Of course, instead of using PCRE_DFA_RESTART, the same technique of re-running -the entire match can also be used with the DFA matching functions. Another -possibility is to work with two buffers. If a partial match at offset <i>n</i> -in the first buffer is followed by "no match" when PCRE_DFA_RESTART is used on -the second buffer, you can then try a new match starting at offset <i>n+1</i> in -the first buffer. -</P> -<br><a name="SEC10" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC11" href="#TOC1">REVISION</a><br> -<P> -Last updated: 02 July 2013 -<br> -Copyright © 1997-2013 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcrepattern.html b/plugins/Pcre16/docs/doc/html/pcrepattern.html deleted file mode 100644 index c06d1e03f1..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcrepattern.html +++ /dev/null @@ -1,3235 +0,0 @@ -<html> -<head> -<title>pcrepattern specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcrepattern man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">PCRE REGULAR EXPRESSION DETAILS</a> -<li><a name="TOC2" href="#SEC2">SPECIAL START-OF-PATTERN ITEMS</a> -<li><a name="TOC3" href="#SEC3">EBCDIC CHARACTER CODES</a> -<li><a name="TOC4" href="#SEC4">CHARACTERS AND METACHARACTERS</a> -<li><a name="TOC5" href="#SEC5">BACKSLASH</a> -<li><a name="TOC6" href="#SEC6">CIRCUMFLEX AND DOLLAR</a> -<li><a name="TOC7" href="#SEC7">FULL STOP (PERIOD, DOT) AND \N</a> -<li><a name="TOC8" href="#SEC8">MATCHING A SINGLE DATA UNIT</a> -<li><a name="TOC9" href="#SEC9">SQUARE BRACKETS AND CHARACTER CLASSES</a> -<li><a name="TOC10" href="#SEC10">POSIX CHARACTER CLASSES</a> -<li><a name="TOC11" href="#SEC11">COMPATIBILITY FEATURE FOR WORD BOUNDARIES</a> -<li><a name="TOC12" href="#SEC12">VERTICAL BAR</a> -<li><a name="TOC13" href="#SEC13">INTERNAL OPTION SETTING</a> -<li><a name="TOC14" href="#SEC14">SUBPATTERNS</a> -<li><a name="TOC15" href="#SEC15">DUPLICATE SUBPATTERN NUMBERS</a> -<li><a name="TOC16" href="#SEC16">NAMED SUBPATTERNS</a> -<li><a name="TOC17" href="#SEC17">REPETITION</a> -<li><a name="TOC18" href="#SEC18">ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS</a> -<li><a name="TOC19" href="#SEC19">BACK REFERENCES</a> -<li><a name="TOC20" href="#SEC20">ASSERTIONS</a> -<li><a name="TOC21" href="#SEC21">CONDITIONAL SUBPATTERNS</a> -<li><a name="TOC22" href="#SEC22">COMMENTS</a> -<li><a name="TOC23" href="#SEC23">RECURSIVE PATTERNS</a> -<li><a name="TOC24" href="#SEC24">SUBPATTERNS AS SUBROUTINES</a> -<li><a name="TOC25" href="#SEC25">ONIGURUMA SUBROUTINE SYNTAX</a> -<li><a name="TOC26" href="#SEC26">CALLOUTS</a> -<li><a name="TOC27" href="#SEC27">BACKTRACKING CONTROL</a> -<li><a name="TOC28" href="#SEC28">SEE ALSO</a> -<li><a name="TOC29" href="#SEC29">AUTHOR</a> -<li><a name="TOC30" href="#SEC30">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">PCRE REGULAR EXPRESSION DETAILS</a><br> -<P> -The syntax and semantics of the regular expressions that are supported by PCRE -are described in detail below. There is a quick-reference syntax summary in the -<a href="pcresyntax.html"><b>pcresyntax</b></a> -page. PCRE tries to match Perl syntax and semantics as closely as it can. PCRE -also supports some alternative regular expression syntax (which does not -conflict with the Perl syntax) in order to provide some compatibility with -regular expressions in Python, .NET, and Oniguruma. -</P> -<P> -Perl's regular expressions are described in its own documentation, and -regular expressions in general are covered in a number of books, some of which -have copious examples. Jeffrey Friedl's "Mastering Regular Expressions", -published by O'Reilly, covers regular expressions in great detail. This -description of PCRE's regular expressions is intended as reference material. -</P> -<P> -This document discusses the patterns that are supported by PCRE when one its -main matching functions, <b>pcre_exec()</b> (8-bit) or <b>pcre[16|32]_exec()</b> -(16- or 32-bit), is used. PCRE also has alternative matching functions, -<b>pcre_dfa_exec()</b> and <b>pcre[16|32_dfa_exec()</b>, which match using a -different algorithm that is not Perl-compatible. Some of the features discussed -below are not available when DFA matching is used. The advantages and -disadvantages of the alternative functions, and how they differ from the normal -functions, are discussed in the -<a href="pcrematching.html"><b>pcrematching</b></a> -page. -</P> -<br><a name="SEC2" href="#TOC1">SPECIAL START-OF-PATTERN ITEMS</a><br> -<P> -A number of options that can be passed to <b>pcre_compile()</b> can also be set -by special items at the start of a pattern. These are not Perl-compatible, but -are provided to make these options accessible to pattern writers who are not -able to change the program that processes the pattern. Any number of these -items may appear, but they must all be together right at the start of the -pattern string, and the letters must be in upper case. -</P> -<br><b> -UTF support -</b><br> -<P> -The original operation of PCRE was on strings of one-byte characters. However, -there is now also support for UTF-8 strings in the original library, an -extra library that supports 16-bit and UTF-16 character strings, and a -third library that supports 32-bit and UTF-32 character strings. To use these -features, PCRE must be built to include appropriate support. When using UTF -strings you must either call the compiling function with the PCRE_UTF8, -PCRE_UTF16, or PCRE_UTF32 option, or the pattern must start with one of -these special sequences: -<pre> - (*UTF8) - (*UTF16) - (*UTF32) - (*UTF) -</pre> -(*UTF) is a generic sequence that can be used with any of the libraries. -Starting a pattern with such a sequence is equivalent to setting the relevant -option. How setting a UTF mode affects pattern matching is mentioned in several -places below. There is also a summary of features in the -<a href="pcreunicode.html"><b>pcreunicode</b></a> -page. -</P> -<P> -Some applications that allow their users to supply patterns may wish to -restrict them to non-UTF data for security reasons. If the PCRE_NEVER_UTF -option is set at compile time, (*UTF) etc. are not allowed, and their -appearance causes an error. -</P> -<br><b> -Unicode property support -</b><br> -<P> -Another special sequence that may appear at the start of a pattern is (*UCP). -This has the same effect as setting the PCRE_UCP option: it causes sequences -such as \d and \w to use Unicode properties to determine character types, -instead of recognizing only characters with codes less than 128 via a lookup -table. -</P> -<br><b> -Disabling auto-possessification -</b><br> -<P> -If a pattern starts with (*NO_AUTO_POSSESS), it has the same effect as setting -the PCRE_NO_AUTO_POSSESS option at compile time. This stops PCRE from making -quantifiers possessive when what follows cannot match the repeated item. For -example, by default a+b is treated as a++b. For more details, see the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation. -</P> -<br><b> -Disabling start-up optimizations -</b><br> -<P> -If a pattern starts with (*NO_START_OPT), it has the same effect as setting the -PCRE_NO_START_OPTIMIZE option either at compile or matching time. This disables -several optimizations for quickly reaching "no match" results. For more -details, see the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation. -<a name="newlines"></a></P> -<br><b> -Newline conventions -</b><br> -<P> -PCRE supports five different conventions for indicating line breaks in -strings: a single CR (carriage return) character, a single LF (linefeed) -character, the two-character sequence CRLF, any of the three preceding, or any -Unicode newline sequence. The -<a href="pcreapi.html"><b>pcreapi</b></a> -page has -<a href="pcreapi.html#newlines">further discussion</a> -about newlines, and shows how to set the newline convention in the -<i>options</i> arguments for the compiling and matching functions. -</P> -<P> -It is also possible to specify a newline convention by starting a pattern -string with one of the following five sequences: -<pre> - (*CR) carriage return - (*LF) linefeed - (*CRLF) carriage return, followed by linefeed - (*ANYCRLF) any of the three above - (*ANY) all Unicode newline sequences -</pre> -These override the default and the options given to the compiling function. For -example, on a Unix system where LF is the default newline sequence, the pattern -<pre> - (*CR)a.b -</pre> -changes the convention to CR. That pattern matches "a\nb" because LF is no -longer a newline. If more than one of these settings is present, the last one -is used. -</P> -<P> -The newline convention affects where the circumflex and dollar assertions are -true. It also affects the interpretation of the dot metacharacter when -PCRE_DOTALL is not set, and the behaviour of \N. However, it does not affect -what the \R escape sequence matches. By default, this is any Unicode newline -sequence, for Perl compatibility. However, this can be changed; see the -description of \R in the section entitled -<a href="#newlineseq">"Newline sequences"</a> -below. A change of \R setting can be combined with a change of newline -convention. -</P> -<br><b> -Setting match and recursion limits -</b><br> -<P> -The caller of <b>pcre_exec()</b> can set a limit on the number of times the -internal <b>match()</b> function is called and on the maximum depth of -recursive calls. These facilities are provided to catch runaway matches that -are provoked by patterns with huge matching trees (a typical example is a -pattern with nested unlimited repeats) and to avoid running out of system stack -by too much recursion. When one of these limits is reached, <b>pcre_exec()</b> -gives an error return. The limits can also be set by items at the start of the -pattern of the form -<pre> - (*LIMIT_MATCH=d) - (*LIMIT_RECURSION=d) -</pre> -where d is any number of decimal digits. However, the value of the setting must -be less than the value set (or defaulted) by the caller of <b>pcre_exec()</b> -for it to have any effect. In other words, the pattern writer can lower the -limits set by the programmer, but not raise them. If there is more than one -setting of one of these limits, the lower value is used. -</P> -<br><a name="SEC3" href="#TOC1">EBCDIC CHARACTER CODES</a><br> -<P> -PCRE can be compiled to run in an environment that uses EBCDIC as its character -code rather than ASCII or Unicode (typically a mainframe system). In the -sections below, character code values are ASCII or Unicode; in an EBCDIC -environment these characters may have different code values, and there are no -code points greater than 255. -</P> -<br><a name="SEC4" href="#TOC1">CHARACTERS AND METACHARACTERS</a><br> -<P> -A regular expression is a pattern that is matched against a subject string from -left to right. Most characters stand for themselves in a pattern, and match the -corresponding characters in the subject. As a trivial example, the pattern -<pre> - The quick brown fox -</pre> -matches a portion of a subject string that is identical to itself. When -caseless matching is specified (the PCRE_CASELESS option), letters are matched -independently of case. In a UTF mode, PCRE always understands the concept of -case for characters whose values are less than 128, so caseless matching is -always possible. For characters with higher values, the concept of case is -supported if PCRE is compiled with Unicode property support, but not otherwise. -If you want to use caseless matching for characters 128 and above, you must -ensure that PCRE is compiled with Unicode property support as well as with -UTF support. -</P> -<P> -The power of regular expressions comes from the ability to include alternatives -and repetitions in the pattern. These are encoded in the pattern by the use of -<i>metacharacters</i>, which do not stand for themselves but instead are -interpreted in some special way. -</P> -<P> -There are two different sets of metacharacters: those that are recognized -anywhere in the pattern except within square brackets, and those that are -recognized within square brackets. Outside square brackets, the metacharacters -are as follows: -<pre> - \ general escape character with several uses - ^ assert start of string (or line, in multiline mode) - $ assert end of string (or line, in multiline mode) - . match any character except newline (by default) - [ start character class definition - | start of alternative branch - ( start subpattern - ) end subpattern - ? extends the meaning of ( - also 0 or 1 quantifier - also quantifier minimizer - * 0 or more quantifier - + 1 or more quantifier - also "possessive quantifier" - { start min/max quantifier -</pre> -Part of a pattern that is in square brackets is called a "character class". In -a character class the only metacharacters are: -<pre> - \ general escape character - ^ negate the class, but only if the first character - - indicates character range - [ POSIX character class (only if followed by POSIX syntax) - ] terminates the character class -</pre> -The following sections describe the use of each of the metacharacters. -</P> -<br><a name="SEC5" href="#TOC1">BACKSLASH</a><br> -<P> -The backslash character has several uses. Firstly, if it is followed by a -character that is not a number or a letter, it takes away any special meaning -that character may have. This use of backslash as an escape character applies -both inside and outside character classes. -</P> -<P> -For example, if you want to match a * character, you write \* in the pattern. -This escaping action applies whether or not the following character would -otherwise be interpreted as a metacharacter, so it is always safe to precede a -non-alphanumeric with backslash to specify that it stands for itself. In -particular, if you want to match a backslash, you write \\. -</P> -<P> -In a UTF mode, only ASCII numbers and letters have any special meaning after a -backslash. All other characters (in particular, those whose codepoints are -greater than 127) are treated as literals. -</P> -<P> -If a pattern is compiled with the PCRE_EXTENDED option, most white space in the -pattern (other than in a character class), and characters between a # outside a -character class and the next newline, inclusive, are ignored. An escaping -backslash can be used to include a white space or # character as part of the -pattern. -</P> -<P> -If you want to remove the special meaning from a sequence of characters, you -can do so by putting them between \Q and \E. This is different from Perl in -that $ and @ are handled as literals in \Q...\E sequences in PCRE, whereas in -Perl, $ and @ cause variable interpolation. Note the following examples: -<pre> - Pattern PCRE matches Perl matches - - \Qabc$xyz\E abc$xyz abc followed by the contents of $xyz - \Qabc\$xyz\E abc\$xyz abc\$xyz - \Qabc\E\$\Qxyz\E abc$xyz abc$xyz -</pre> -The \Q...\E sequence is recognized both inside and outside character classes. -An isolated \E that is not preceded by \Q is ignored. If \Q is not followed -by \E later in the pattern, the literal interpretation continues to the end of -the pattern (that is, \E is assumed at the end). If the isolated \Q is inside -a character class, this causes an error, because the character class is not -terminated. -<a name="digitsafterbackslash"></a></P> -<br><b> -Non-printing characters -</b><br> -<P> -A second use of backslash provides a way of encoding non-printing characters -in patterns in a visible manner. There is no restriction on the appearance of -non-printing characters, apart from the binary zero that terminates a pattern, -but when a pattern is being prepared by text editing, it is often easier to use -one of the following escape sequences than the binary character it represents: -<pre> - \a alarm, that is, the BEL character (hex 07) - \cx "control-x", where x is any ASCII character - \e escape (hex 1B) - \f form feed (hex 0C) - \n linefeed (hex 0A) - \r carriage return (hex 0D) - \t tab (hex 09) - \0dd character with octal code 0dd - \ddd character with octal code ddd, or back reference - \o{ddd..} character with octal code ddd.. - \xhh character with hex code hh - \x{hhh..} character with hex code hhh.. (non-JavaScript mode) - \uhhhh character with hex code hhhh (JavaScript mode only) -</pre> -The precise effect of \cx on ASCII characters is as follows: if x is a lower -case letter, it is converted to upper case. Then bit 6 of the character (hex -40) is inverted. Thus \cA to \cZ become hex 01 to hex 1A (A is 41, Z is 5A), -but \c{ becomes hex 3B ({ is 7B), and \c; becomes hex 7B (; is 3B). If the -data item (byte or 16-bit value) following \c has a value greater than 127, a -compile-time error occurs. This locks out non-ASCII characters in all modes. -</P> -<P> -The \c facility was designed for use with ASCII characters, but with the -extension to Unicode it is even less useful than it once was. It is, however, -recognized when PCRE is compiled in EBCDIC mode, where data items are always -bytes. In this mode, all values are valid after \c. If the next character is a -lower case letter, it is converted to upper case. Then the 0xc0 bits of the -byte are inverted. Thus \cA becomes hex 01, as in ASCII (A is C1), but because -the EBCDIC letters are disjoint, \cZ becomes hex 29 (Z is E9), and other -characters also generate different values. -</P> -<P> -After \0 up to two further octal digits are read. If there are fewer than two -digits, just those that are present are used. Thus the sequence \0\x\07 -specifies two binary zeros followed by a BEL character (code value 7). Make -sure you supply two digits after the initial zero if the pattern character that -follows is itself an octal digit. -</P> -<P> -The escape \o must be followed by a sequence of octal digits, enclosed in -braces. An error occurs if this is not the case. This escape is a recent -addition to Perl; it provides way of specifying character code points as octal -numbers greater than 0777, and it also allows octal numbers and back references -to be unambiguously specified. -</P> -<P> -For greater clarity and unambiguity, it is best to avoid following \ by a -digit greater than zero. Instead, use \o{} or \x{} to specify character -numbers, and \g{} to specify back references. The following paragraphs -describe the old, ambiguous syntax. -</P> -<P> -The handling of a backslash followed by a digit other than 0 is complicated, -and Perl has changed in recent releases, causing PCRE also to change. Outside a -character class, PCRE reads the digit and any following digits as a decimal -number. If the number is less than 8, or if there have been at least that many -previous capturing left parentheses in the expression, the entire sequence is -taken as a <i>back reference</i>. A description of how this works is given -<a href="#backreferences">later,</a> -following the discussion of -<a href="#subpattern">parenthesized subpatterns.</a> -</P> -<P> -Inside a character class, or if the decimal number following \ is greater than -7 and there have not been that many capturing subpatterns, PCRE handles \8 and -\9 as the literal characters "8" and "9", and otherwise re-reads up to three -octal digits following the backslash, using them to generate a data character. -Any subsequent digits stand for themselves. For example: -<pre> - \040 is another way of writing an ASCII space - \40 is the same, provided there are fewer than 40 previous capturing subpatterns - \7 is always a back reference - \11 might be a back reference, or another way of writing a tab - \011 is always a tab - \0113 is a tab followed by the character "3" - \113 might be a back reference, otherwise the character with octal code 113 - \377 might be a back reference, otherwise the value 255 (decimal) - \81 is either a back reference, or the two characters "8" and "1" -</pre> -Note that octal values of 100 or greater that are specified using this syntax -must not be introduced by a leading zero, because no more than three octal -digits are ever read. -</P> -<P> -By default, after \x that is not followed by {, from zero to two hexadecimal -digits are read (letters can be in upper or lower case). Any number of -hexadecimal digits may appear between \x{ and }. If a character other than -a hexadecimal digit appears between \x{ and }, or if there is no terminating -}, an error occurs. -</P> -<P> -If the PCRE_JAVASCRIPT_COMPAT option is set, the interpretation of \x is -as just described only when it is followed by two hexadecimal digits. -Otherwise, it matches a literal "x" character. In JavaScript mode, support for -code points greater than 256 is provided by \u, which must be followed by -four hexadecimal digits; otherwise it matches a literal "u" character. -</P> -<P> -Characters whose value is less than 256 can be defined by either of the two -syntaxes for \x (or by \u in JavaScript mode). There is no difference in the -way they are handled. For example, \xdc is exactly the same as \x{dc} (or -\u00dc in JavaScript mode). -</P> -<br><b> -Constraints on character values -</b><br> -<P> -Characters that are specified using octal or hexadecimal numbers are -limited to certain values, as follows: -<pre> - 8-bit non-UTF mode less than 0x100 - 8-bit UTF-8 mode less than 0x10ffff and a valid codepoint - 16-bit non-UTF mode less than 0x10000 - 16-bit UTF-16 mode less than 0x10ffff and a valid codepoint - 32-bit non-UTF mode less than 0x100000000 - 32-bit UTF-32 mode less than 0x10ffff and a valid codepoint -</pre> -Invalid Unicode codepoints are the range 0xd800 to 0xdfff (the so-called -"surrogate" codepoints), and 0xffef. -</P> -<br><b> -Escape sequences in character classes -</b><br> -<P> -All the sequences that define a single character value can be used both inside -and outside character classes. In addition, inside a character class, \b is -interpreted as the backspace character (hex 08). -</P> -<P> -\N is not allowed in a character class. \B, \R, and \X are not special -inside a character class. Like other unrecognized escape sequences, they are -treated as the literal characters "B", "R", and "X" by default, but cause an -error if the PCRE_EXTRA option is set. Outside a character class, these -sequences have different meanings. -</P> -<br><b> -Unsupported escape sequences -</b><br> -<P> -In Perl, the sequences \l, \L, \u, and \U are recognized by its string -handler and used to modify the case of following characters. By default, PCRE -does not support these escape sequences. However, if the PCRE_JAVASCRIPT_COMPAT -option is set, \U matches a "U" character, and \u can be used to define a -character by code point, as described in the previous section. -</P> -<br><b> -Absolute and relative back references -</b><br> -<P> -The sequence \g followed by an unsigned or a negative number, optionally -enclosed in braces, is an absolute or relative back reference. A named back -reference can be coded as \g{name}. Back references are discussed -<a href="#backreferences">later,</a> -following the discussion of -<a href="#subpattern">parenthesized subpatterns.</a> -</P> -<br><b> -Absolute and relative subroutine calls -</b><br> -<P> -For compatibility with Oniguruma, the non-Perl syntax \g followed by a name or -a number enclosed either in angle brackets or single quotes, is an alternative -syntax for referencing a subpattern as a "subroutine". Details are discussed -<a href="#onigurumasubroutines">later.</a> -Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are <i>not</i> -synonymous. The former is a back reference; the latter is a -<a href="#subpatternsassubroutines">subroutine</a> -call. -<a name="genericchartypes"></a></P> -<br><b> -Generic character types -</b><br> -<P> -Another use of backslash is for specifying generic character types: -<pre> - \d any decimal digit - \D any character that is not a decimal digit - \h any horizontal white space character - \H any character that is not a horizontal white space character - \s any white space character - \S any character that is not a white space character - \v any vertical white space character - \V any character that is not a vertical white space character - \w any "word" character - \W any "non-word" character -</pre> -There is also the single sequence \N, which matches a non-newline character. -This is the same as -<a href="#fullstopdot">the "." metacharacter</a> -when PCRE_DOTALL is not set. Perl also uses \N to match characters by name; -PCRE does not support this. -</P> -<P> -Each pair of lower and upper case escape sequences partitions the complete set -of characters into two disjoint sets. Any given character matches one, and only -one, of each pair. The sequences can appear both inside and outside character -classes. They each match one character of the appropriate type. If the current -matching point is at the end of the subject string, all of them fail, because -there is no character to match. -</P> -<P> -For compatibility with Perl, \s did not used to match the VT character (code -11), which made it different from the the POSIX "space" class. However, Perl -added VT at release 5.18, and PCRE followed suit at release 8.34. The default -\s characters are now HT (9), LF (10), VT (11), FF (12), CR (13), and space -(32), which are defined as white space in the "C" locale. This list may vary if -locale-specific matching is taking place. For example, in some locales the -"non-breaking space" character (\xA0) is recognized as white space, and in -others the VT character is not. -</P> -<P> -A "word" character is an underscore or any character that is a letter or digit. -By default, the definition of letters and digits is controlled by PCRE's -low-valued character tables, and may vary if locale-specific matching is taking -place (see -<a href="pcreapi.html#localesupport">"Locale support"</a> -in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page). For example, in a French locale such as "fr_FR" in Unix-like systems, -or "french" in Windows, some character codes greater than 127 are used for -accented letters, and these are then matched by \w. The use of locales with -Unicode is discouraged. -</P> -<P> -By default, characters whose code points are greater than 127 never match \d, -\s, or \w, and always match \D, \S, and \W, although this may vary for -characters in the range 128-255 when locale-specific matching is happening. -These escape sequences retain their original meanings from before Unicode -support was available, mainly for efficiency reasons. If PCRE is compiled with -Unicode property support, and the PCRE_UCP option is set, the behaviour is -changed so that Unicode properties are used to determine character types, as -follows: -<pre> - \d any character that matches \p{Nd} (decimal digit) - \s any character that matches \p{Z} or \h or \v - \w any character that matches \p{L} or \p{N}, plus underscore -</pre> -The upper case escapes match the inverse sets of characters. Note that \d -matches only decimal digits, whereas \w matches any Unicode digit, as well as -any Unicode letter, and underscore. Note also that PCRE_UCP affects \b, and -\B because they are defined in terms of \w and \W. Matching these sequences -is noticeably slower when PCRE_UCP is set. -</P> -<P> -The sequences \h, \H, \v, and \V are features that were added to Perl at -release 5.10. In contrast to the other sequences, which match only ASCII -characters by default, these always match certain high-valued code points, -whether or not PCRE_UCP is set. The horizontal space characters are: -<pre> - U+0009 Horizontal tab (HT) - U+0020 Space - U+00A0 Non-break space - U+1680 Ogham space mark - U+180E Mongolian vowel separator - U+2000 En quad - U+2001 Em quad - U+2002 En space - U+2003 Em space - U+2004 Three-per-em space - U+2005 Four-per-em space - U+2006 Six-per-em space - U+2007 Figure space - U+2008 Punctuation space - U+2009 Thin space - U+200A Hair space - U+202F Narrow no-break space - U+205F Medium mathematical space - U+3000 Ideographic space -</pre> -The vertical space characters are: -<pre> - U+000A Linefeed (LF) - U+000B Vertical tab (VT) - U+000C Form feed (FF) - U+000D Carriage return (CR) - U+0085 Next line (NEL) - U+2028 Line separator - U+2029 Paragraph separator -</pre> -In 8-bit, non-UTF-8 mode, only the characters with codepoints less than 256 are -relevant. -<a name="newlineseq"></a></P> -<br><b> -Newline sequences -</b><br> -<P> -Outside a character class, by default, the escape sequence \R matches any -Unicode newline sequence. In 8-bit non-UTF-8 mode \R is equivalent to the -following: -<pre> - (?>\r\n|\n|\x0b|\f|\r|\x85) -</pre> -This is an example of an "atomic group", details of which are given -<a href="#atomicgroup">below.</a> -This particular group matches either the two-character sequence CR followed by -LF, or one of the single characters LF (linefeed, U+000A), VT (vertical tab, -U+000B), FF (form feed, U+000C), CR (carriage return, U+000D), or NEL (next -line, U+0085). The two-character sequence is treated as a single unit that -cannot be split. -</P> -<P> -In other modes, two additional characters whose codepoints are greater than 255 -are added: LS (line separator, U+2028) and PS (paragraph separator, U+2029). -Unicode character property support is not needed for these characters to be -recognized. -</P> -<P> -It is possible to restrict \R to match only CR, LF, or CRLF (instead of the -complete set of Unicode line endings) by setting the option PCRE_BSR_ANYCRLF -either at compile time or when the pattern is matched. (BSR is an abbrevation -for "backslash R".) This can be made the default when PCRE is built; if this is -the case, the other behaviour can be requested via the PCRE_BSR_UNICODE option. -It is also possible to specify these settings by starting a pattern string with -one of the following sequences: -<pre> - (*BSR_ANYCRLF) CR, LF, or CRLF only - (*BSR_UNICODE) any Unicode newline sequence -</pre> -These override the default and the options given to the compiling function, but -they can themselves be overridden by options given to a matching function. Note -that these special settings, which are not Perl-compatible, are recognized only -at the very start of a pattern, and that they must be in upper case. If more -than one of them is present, the last one is used. They can be combined with a -change of newline convention; for example, a pattern can start with: -<pre> - (*ANY)(*BSR_ANYCRLF) -</pre> -They can also be combined with the (*UTF8), (*UTF16), (*UTF32), (*UTF) or -(*UCP) special sequences. Inside a character class, \R is treated as an -unrecognized escape sequence, and so matches the letter "R" by default, but -causes an error if PCRE_EXTRA is set. -<a name="uniextseq"></a></P> -<br><b> -Unicode character properties -</b><br> -<P> -When PCRE is built with Unicode character property support, three additional -escape sequences that match characters with specific properties are available. -When in 8-bit non-UTF-8 mode, these sequences are of course limited to testing -characters whose codepoints are less than 256, but they do work in this mode. -The extra escape sequences are: -<pre> - \p{<i>xx</i>} a character with the <i>xx</i> property - \P{<i>xx</i>} a character without the <i>xx</i> property - \X a Unicode extended grapheme cluster -</pre> -The property names represented by <i>xx</i> above are limited to the Unicode -script names, the general category properties, "Any", which matches any -character (including newline), and some special PCRE properties (described -in the -<a href="#extraprops">next section).</a> -Other Perl properties such as "InMusicalSymbols" are not currently supported by -PCRE. Note that \P{Any} does not match any characters, so always causes a -match failure. -</P> -<P> -Sets of Unicode characters are defined as belonging to certain scripts. A -character from one of these sets can be matched using a script name. For -example: -<pre> - \p{Greek} - \P{Han} -</pre> -Those that are not part of an identified script are lumped together as -"Common". The current list of scripts is: -</P> -<P> -Arabic, -Armenian, -Avestan, -Balinese, -Bamum, -Batak, -Bengali, -Bopomofo, -Brahmi, -Braille, -Buginese, -Buhid, -Canadian_Aboriginal, -Carian, -Chakma, -Cham, -Cherokee, -Common, -Coptic, -Cuneiform, -Cypriot, -Cyrillic, -Deseret, -Devanagari, -Egyptian_Hieroglyphs, -Ethiopic, -Georgian, -Glagolitic, -Gothic, -Greek, -Gujarati, -Gurmukhi, -Han, -Hangul, -Hanunoo, -Hebrew, -Hiragana, -Imperial_Aramaic, -Inherited, -Inscriptional_Pahlavi, -Inscriptional_Parthian, -Javanese, -Kaithi, -Kannada, -Katakana, -Kayah_Li, -Kharoshthi, -Khmer, -Lao, -Latin, -Lepcha, -Limbu, -Linear_B, -Lisu, -Lycian, -Lydian, -Malayalam, -Mandaic, -Meetei_Mayek, -Meroitic_Cursive, -Meroitic_Hieroglyphs, -Miao, -Mongolian, -Myanmar, -New_Tai_Lue, -Nko, -Ogham, -Old_Italic, -Old_Persian, -Old_South_Arabian, -Old_Turkic, -Ol_Chiki, -Oriya, -Osmanya, -Phags_Pa, -Phoenician, -Rejang, -Runic, -Samaritan, -Saurashtra, -Sharada, -Shavian, -Sinhala, -Sora_Sompeng, -Sundanese, -Syloti_Nagri, -Syriac, -Tagalog, -Tagbanwa, -Tai_Le, -Tai_Tham, -Tai_Viet, -Takri, -Tamil, -Telugu, -Thaana, -Thai, -Tibetan, -Tifinagh, -Ugaritic, -Vai, -Yi. -</P> -<P> -Each character has exactly one Unicode general category property, specified by -a two-letter abbreviation. For compatibility with Perl, negation can be -specified by including a circumflex between the opening brace and the property -name. For example, \p{^Lu} is the same as \P{Lu}. -</P> -<P> -If only one letter is specified with \p or \P, it includes all the general -category properties that start with that letter. In this case, in the absence -of negation, the curly brackets in the escape sequence are optional; these two -examples have the same effect: -<pre> - \p{L} - \pL -</pre> -The following general category property codes are supported: -<pre> - C Other - Cc Control - Cf Format - Cn Unassigned - Co Private use - Cs Surrogate - - L Letter - Ll Lower case letter - Lm Modifier letter - Lo Other letter - Lt Title case letter - Lu Upper case letter - - M Mark - Mc Spacing mark - Me Enclosing mark - Mn Non-spacing mark - - N Number - Nd Decimal number - Nl Letter number - No Other number - - P Punctuation - Pc Connector punctuation - Pd Dash punctuation - Pe Close punctuation - Pf Final punctuation - Pi Initial punctuation - Po Other punctuation - Ps Open punctuation - - S Symbol - Sc Currency symbol - Sk Modifier symbol - Sm Mathematical symbol - So Other symbol - - Z Separator - Zl Line separator - Zp Paragraph separator - Zs Space separator -</pre> -The special property L& is also supported: it matches a character that has -the Lu, Ll, or Lt property, in other words, a letter that is not classified as -a modifier or "other". -</P> -<P> -The Cs (Surrogate) property applies only to characters in the range U+D800 to -U+DFFF. Such characters are not valid in Unicode strings and so -cannot be tested by PCRE, unless UTF validity checking has been turned off -(see the discussion of PCRE_NO_UTF8_CHECK, PCRE_NO_UTF16_CHECK and -PCRE_NO_UTF32_CHECK in the -<a href="pcreapi.html"><b>pcreapi</b></a> -page). Perl does not support the Cs property. -</P> -<P> -The long synonyms for property names that Perl supports (such as \p{Letter}) -are not supported by PCRE, nor is it permitted to prefix any of these -properties with "Is". -</P> -<P> -No character that is in the Unicode table has the Cn (unassigned) property. -Instead, this property is assumed for any code point that is not in the -Unicode table. -</P> -<P> -Specifying caseless matching does not affect these escape sequences. For -example, \p{Lu} always matches only upper case letters. This is different from -the behaviour of current versions of Perl. -</P> -<P> -Matching characters by Unicode property is not fast, because PCRE has to do a -multistage table lookup in order to find a character's property. That is why -the traditional escape sequences such as \d and \w do not use Unicode -properties in PCRE by default, though you can make them do so by setting the -PCRE_UCP option or by starting the pattern with (*UCP). -</P> -<br><b> -Extended grapheme clusters -</b><br> -<P> -The \X escape matches any number of Unicode characters that form an "extended -grapheme cluster", and treats the sequence as an atomic group -<a href="#atomicgroup">(see below).</a> -Up to and including release 8.31, PCRE matched an earlier, simpler definition -that was equivalent to -<pre> - (?>\PM\pM*) -</pre> -That is, it matched a character without the "mark" property, followed by zero -or more characters with the "mark" property. Characters with the "mark" -property are typically non-spacing accents that affect the preceding character. -</P> -<P> -This simple definition was extended in Unicode to include more complicated -kinds of composite character by giving each character a grapheme breaking -property, and creating rules that use these properties to define the boundaries -of extended grapheme clusters. In releases of PCRE later than 8.31, \X matches -one of these clusters. -</P> -<P> -\X always matches at least one character. Then it decides whether to add -additional characters according to the following rules for ending a cluster: -</P> -<P> -1. End at the end of the subject string. -</P> -<P> -2. Do not end between CR and LF; otherwise end after any control character. -</P> -<P> -3. Do not break Hangul (a Korean script) syllable sequences. Hangul characters -are of five types: L, V, T, LV, and LVT. An L character may be followed by an -L, V, LV, or LVT character; an LV or V character may be followed by a V or T -character; an LVT or T character may be follwed only by a T character. -</P> -<P> -4. Do not end before extending characters or spacing marks. Characters with -the "mark" property always have the "extend" grapheme breaking property. -</P> -<P> -5. Do not end after prepend characters. -</P> -<P> -6. Otherwise, end the cluster. -<a name="extraprops"></a></P> -<br><b> -PCRE's additional properties -</b><br> -<P> -As well as the standard Unicode properties described above, PCRE supports four -more that make it possible to convert traditional escape sequences such as \w -and \s to use Unicode properties. PCRE uses these non-standard, non-Perl -properties internally when PCRE_UCP is set. However, they may also be used -explicitly. These properties are: -<pre> - Xan Any alphanumeric character - Xps Any POSIX space character - Xsp Any Perl space character - Xwd Any Perl "word" character -</pre> -Xan matches characters that have either the L (letter) or the N (number) -property. Xps matches the characters tab, linefeed, vertical tab, form feed, or -carriage return, and any other character that has the Z (separator) property. -Xsp is the same as Xps; it used to exclude vertical tab, for Perl -compatibility, but Perl changed, and so PCRE followed at release 8.34. Xwd -matches the same characters as Xan, plus underscore. -</P> -<P> -There is another non-standard property, Xuc, which matches any character that -can be represented by a Universal Character Name in C++ and other programming -languages. These are the characters $, @, ` (grave accent), and all characters -with Unicode code points greater than or equal to U+00A0, except for the -surrogates U+D800 to U+DFFF. Note that most base (ASCII) characters are -excluded. (Universal Character Names are of the form \uHHHH or \UHHHHHHHH -where H is a hexadecimal digit. Note that the Xuc property does not match these -sequences but the characters that they represent.) -<a name="resetmatchstart"></a></P> -<br><b> -Resetting the match start -</b><br> -<P> -The escape sequence \K causes any previously matched characters not to be -included in the final matched sequence. For example, the pattern: -<pre> - foo\Kbar -</pre> -matches "foobar", but reports that it has matched "bar". This feature is -similar to a lookbehind assertion -<a href="#lookbehind">(described below).</a> -However, in this case, the part of the subject before the real match does not -have to be of fixed length, as lookbehind assertions do. The use of \K does -not interfere with the setting of -<a href="#subpattern">captured substrings.</a> -For example, when the pattern -<pre> - (foo)\Kbar -</pre> -matches "foobar", the first substring is still set to "foo". -</P> -<P> -Perl documents that the use of \K within assertions is "not well defined". In -PCRE, \K is acted upon when it occurs inside positive assertions, but is -ignored in negative assertions. Note that when a pattern such as (?=ab\K) -matches, the reported start of the match can be greater than the end of the -match. -<a name="smallassertions"></a></P> -<br><b> -Simple assertions -</b><br> -<P> -The final use of backslash is for certain simple assertions. An assertion -specifies a condition that has to be met at a particular point in a match, -without consuming any characters from the subject string. The use of -subpatterns for more complicated assertions is described -<a href="#bigassertions">below.</a> -The backslashed assertions are: -<pre> - \b matches at a word boundary - \B matches when not at a word boundary - \A matches at the start of the subject - \Z matches at the end of the subject - also matches before a newline at the end of the subject - \z matches only at the end of the subject - \G matches at the first matching position in the subject -</pre> -Inside a character class, \b has a different meaning; it matches the backspace -character. If any other of these assertions appears in a character class, by -default it matches the corresponding literal character (for example, \B -matches the letter B). However, if the PCRE_EXTRA option is set, an "invalid -escape sequence" error is generated instead. -</P> -<P> -A word boundary is a position in the subject string where the current character -and the previous character do not both match \w or \W (i.e. one matches -\w and the other matches \W), or the start or end of the string if the -first or last character matches \w, respectively. In a UTF mode, the meanings -of \w and \W can be changed by setting the PCRE_UCP option. When this is -done, it also affects \b and \B. Neither PCRE nor Perl has a separate "start -of word" or "end of word" metasequence. However, whatever follows \b normally -determines which it is. For example, the fragment \ba matches "a" at the start -of a word. -</P> -<P> -The \A, \Z, and \z assertions differ from the traditional circumflex and -dollar (described in the next section) in that they only ever match at the very -start and end of the subject string, whatever options are set. Thus, they are -independent of multiline mode. These three assertions are not affected by the -PCRE_NOTBOL or PCRE_NOTEOL options, which affect only the behaviour of the -circumflex and dollar metacharacters. However, if the <i>startoffset</i> -argument of <b>pcre_exec()</b> is non-zero, indicating that matching is to start -at a point other than the beginning of the subject, \A can never match. The -difference between \Z and \z is that \Z matches before a newline at the end -of the string as well as at the very end, whereas \z matches only at the end. -</P> -<P> -The \G assertion is true only when the current matching position is at the -start point of the match, as specified by the <i>startoffset</i> argument of -<b>pcre_exec()</b>. It differs from \A when the value of <i>startoffset</i> is -non-zero. By calling <b>pcre_exec()</b> multiple times with appropriate -arguments, you can mimic Perl's /g option, and it is in this kind of -implementation where \G can be useful. -</P> -<P> -Note, however, that PCRE's interpretation of \G, as the start of the current -match, is subtly different from Perl's, which defines it as the end of the -previous match. In Perl, these can be different when the previously matched -string was empty. Because PCRE does just one match at a time, it cannot -reproduce this behaviour. -</P> -<P> -If all the alternatives of a pattern begin with \G, the expression is anchored -to the starting match position, and the "anchored" flag is set in the compiled -regular expression. -</P> -<br><a name="SEC6" href="#TOC1">CIRCUMFLEX AND DOLLAR</a><br> -<P> -The circumflex and dollar metacharacters are zero-width assertions. That is, -they test for a particular condition being true without consuming any -characters from the subject string. -</P> -<P> -Outside a character class, in the default matching mode, the circumflex -character is an assertion that is true only if the current matching point is at -the start of the subject string. If the <i>startoffset</i> argument of -<b>pcre_exec()</b> is non-zero, circumflex can never match if the PCRE_MULTILINE -option is unset. Inside a character class, circumflex has an entirely different -meaning -<a href="#characterclass">(see below).</a> -</P> -<P> -Circumflex need not be the first character of the pattern if a number of -alternatives are involved, but it should be the first thing in each alternative -in which it appears if the pattern is ever to match that branch. If all -possible alternatives start with a circumflex, that is, if the pattern is -constrained to match only at the start of the subject, it is said to be an -"anchored" pattern. (There are also other constructs that can cause a pattern -to be anchored.) -</P> -<P> -The dollar character is an assertion that is true only if the current matching -point is at the end of the subject string, or immediately before a newline at -the end of the string (by default). Note, however, that it does not actually -match the newline. Dollar need not be the last character of the pattern if a -number of alternatives are involved, but it should be the last item in any -branch in which it appears. Dollar has no special meaning in a character class. -</P> -<P> -The meaning of dollar can be changed so that it matches only at the very end of -the string, by setting the PCRE_DOLLAR_ENDONLY option at compile time. This -does not affect the \Z assertion. -</P> -<P> -The meanings of the circumflex and dollar characters are changed if the -PCRE_MULTILINE option is set. When this is the case, a circumflex matches -immediately after internal newlines as well as at the start of the subject -string. It does not match after a newline that ends the string. A dollar -matches before any newlines in the string, as well as at the very end, when -PCRE_MULTILINE is set. When newline is specified as the two-character -sequence CRLF, isolated CR and LF characters do not indicate newlines. -</P> -<P> -For example, the pattern /^abc$/ matches the subject string "def\nabc" (where -\n represents a newline) in multiline mode, but not otherwise. Consequently, -patterns that are anchored in single line mode because all branches start with -^ are not anchored in multiline mode, and a match for circumflex is possible -when the <i>startoffset</i> argument of <b>pcre_exec()</b> is non-zero. The -PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is set. -</P> -<P> -Note that the sequences \A, \Z, and \z can be used to match the start and -end of the subject in both modes, and if all branches of a pattern start with -\A it is always anchored, whether or not PCRE_MULTILINE is set. -<a name="fullstopdot"></a></P> -<br><a name="SEC7" href="#TOC1">FULL STOP (PERIOD, DOT) AND \N</a><br> -<P> -Outside a character class, a dot in the pattern matches any one character in -the subject string except (by default) a character that signifies the end of a -line. -</P> -<P> -When a line ending is defined as a single character, dot never matches that -character; when the two-character sequence CRLF is used, dot does not match CR -if it is immediately followed by LF, but otherwise it matches all characters -(including isolated CRs and LFs). When any Unicode line endings are being -recognized, dot does not match CR or LF or any of the other line ending -characters. -</P> -<P> -The behaviour of dot with regard to newlines can be changed. If the PCRE_DOTALL -option is set, a dot matches any one character, without exception. If the -two-character sequence CRLF is present in the subject string, it takes two dots -to match it. -</P> -<P> -The handling of dot is entirely independent of the handling of circumflex and -dollar, the only relationship being that they both involve newlines. Dot has no -special meaning in a character class. -</P> -<P> -The escape sequence \N behaves like a dot, except that it is not affected by -the PCRE_DOTALL option. In other words, it matches any character except one -that signifies the end of a line. Perl also uses \N to match characters by -name; PCRE does not support this. -</P> -<br><a name="SEC8" href="#TOC1">MATCHING A SINGLE DATA UNIT</a><br> -<P> -Outside a character class, the escape sequence \C matches any one data unit, -whether or not a UTF mode is set. In the 8-bit library, one data unit is one -byte; in the 16-bit library it is a 16-bit unit; in the 32-bit library it is -a 32-bit unit. Unlike a dot, \C always -matches line-ending characters. The feature is provided in Perl in order to -match individual bytes in UTF-8 mode, but it is unclear how it can usefully be -used. Because \C breaks up characters into individual data units, matching one -unit with \C in a UTF mode means that the rest of the string may start with a -malformed UTF character. This has undefined results, because PCRE assumes that -it is dealing with valid UTF strings (and by default it checks this at the -start of processing unless the PCRE_NO_UTF8_CHECK, PCRE_NO_UTF16_CHECK or -PCRE_NO_UTF32_CHECK option is used). -</P> -<P> -PCRE does not allow \C to appear in lookbehind assertions -<a href="#lookbehind">(described below)</a> -in a UTF mode, because this would make it impossible to calculate the length of -the lookbehind. -</P> -<P> -In general, the \C escape sequence is best avoided. However, one -way of using it that avoids the problem of malformed UTF characters is to use a -lookahead to check the length of the next character, as in this pattern, which -could be used with a UTF-8 string (ignore white space and line breaks): -<pre> - (?| (?=[\x00-\x7f])(\C) | - (?=[\x80-\x{7ff}])(\C)(\C) | - (?=[\x{800}-\x{ffff}])(\C)(\C)(\C) | - (?=[\x{10000}-\x{1fffff}])(\C)(\C)(\C)(\C)) -</pre> -A group that starts with (?| resets the capturing parentheses numbers in each -alternative (see -<a href="#dupsubpatternnumber">"Duplicate Subpattern Numbers"</a> -below). The assertions at the start of each branch check the next UTF-8 -character for values whose encoding uses 1, 2, 3, or 4 bytes, respectively. The -character's individual bytes are then captured by the appropriate number of -groups. -<a name="characterclass"></a></P> -<br><a name="SEC9" href="#TOC1">SQUARE BRACKETS AND CHARACTER CLASSES</a><br> -<P> -An opening square bracket introduces a character class, terminated by a closing -square bracket. A closing square bracket on its own is not special by default. -However, if the PCRE_JAVASCRIPT_COMPAT option is set, a lone closing square -bracket causes a compile-time error. If a closing square bracket is required as -a member of the class, it should be the first data character in the class -(after an initial circumflex, if present) or escaped with a backslash. -</P> -<P> -A character class matches a single character in the subject. In a UTF mode, the -character may be more than one data unit long. A matched character must be in -the set of characters defined by the class, unless the first character in the -class definition is a circumflex, in which case the subject character must not -be in the set defined by the class. If a circumflex is actually required as a -member of the class, ensure it is not the first character, or escape it with a -backslash. -</P> -<P> -For example, the character class [aeiou] matches any lower case vowel, while -[^aeiou] matches any character that is not a lower case vowel. Note that a -circumflex is just a convenient notation for specifying the characters that -are in the class by enumerating those that are not. A class that starts with a -circumflex is not an assertion; it still consumes a character from the subject -string, and therefore it fails if the current pointer is at the end of the -string. -</P> -<P> -In UTF-8 (UTF-16, UTF-32) mode, characters with values greater than 255 (0xffff) -can be included in a class as a literal string of data units, or by using the -\x{ escaping mechanism. -</P> -<P> -When caseless matching is set, any letters in a class represent both their -upper case and lower case versions, so for example, a caseless [aeiou] matches -"A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a -caseful version would. In a UTF mode, PCRE always understands the concept of -case for characters whose values are less than 128, so caseless matching is -always possible. For characters with higher values, the concept of case is -supported if PCRE is compiled with Unicode property support, but not otherwise. -If you want to use caseless matching in a UTF mode for characters 128 and -above, you must ensure that PCRE is compiled with Unicode property support as -well as with UTF support. -</P> -<P> -Characters that might indicate line breaks are never treated in any special way -when matching character classes, whatever line-ending sequence is in use, and -whatever setting of the PCRE_DOTALL and PCRE_MULTILINE options is used. A class -such as [^a] always matches one of these characters. -</P> -<P> -The minus (hyphen) character can be used to specify a range of characters in a -character class. For example, [d-m] matches any letter between d and m, -inclusive. If a minus character is required in a class, it must be escaped with -a backslash or appear in a position where it cannot be interpreted as -indicating a range, typically as the first or last character in the class, or -immediately after a range. For example, [b-d-z] matches letters in the range b -to d, a hyphen character, or z. -</P> -<P> -It is not possible to have the literal character "]" as the end character of a -range. A pattern such as [W-]46] is interpreted as a class of two characters -("W" and "-") followed by a literal string "46]", so it would match "W46]" or -"-46]". However, if the "]" is escaped with a backslash it is interpreted as -the end of range, so [W-\]46] is interpreted as a class containing a range -followed by two other characters. The octal or hexadecimal representation of -"]" can also be used to end a range. -</P> -<P> -An error is generated if a POSIX character class (see below) or an escape -sequence other than one that defines a single character appears at a point -where a range ending character is expected. For example, [z-\xff] is valid, -but [A-\d] and [A-[:digit:]] are not. -</P> -<P> -Ranges operate in the collating sequence of character values. They can also be -used for characters specified numerically, for example [\000-\037]. Ranges -can include any characters that are valid for the current mode. -</P> -<P> -If a range that includes letters is used when caseless matching is set, it -matches the letters in either case. For example, [W-c] is equivalent to -[][\\^_`wxyzabc], matched caselessly, and in a non-UTF mode, if character -tables for a French locale are in use, [\xc8-\xcb] matches accented E -characters in both cases. In UTF modes, PCRE supports the concept of case for -characters with values greater than 128 only when it is compiled with Unicode -property support. -</P> -<P> -The character escape sequences \d, \D, \h, \H, \p, \P, \s, \S, \v, -\V, \w, and \W may appear in a character class, and add the characters that -they match to the class. For example, [\dABCDEF] matches any hexadecimal -digit. In UTF modes, the PCRE_UCP option affects the meanings of \d, \s, \w -and their upper case partners, just as it does when they appear outside a -character class, as described in the section entitled -<a href="#genericchartypes">"Generic character types"</a> -above. The escape sequence \b has a different meaning inside a character -class; it matches the backspace character. The sequences \B, \N, \R, and \X -are not special inside a character class. Like any other unrecognized escape -sequences, they are treated as the literal characters "B", "N", "R", and "X" by -default, but cause an error if the PCRE_EXTRA option is set. -</P> -<P> -A circumflex can conveniently be used with the upper case character types to -specify a more restricted set of characters than the matching lower case type. -For example, the class [^\W_] matches any letter or digit, but not underscore, -whereas [\w] includes underscore. A positive character class should be read as -"something OR something OR ..." and a negative class as "NOT something AND NOT -something AND NOT ...". -</P> -<P> -The only metacharacters that are recognized in character classes are backslash, -hyphen (only where it can be interpreted as specifying a range), circumflex -(only at the start), opening square bracket (only when it can be interpreted as -introducing a POSIX class name, or for a special compatibility feature - see -the next two sections), and the terminating closing square bracket. However, -escaping other non-alphanumeric characters does no harm. -</P> -<br><a name="SEC10" href="#TOC1">POSIX CHARACTER CLASSES</a><br> -<P> -Perl supports the POSIX notation for character classes. This uses names -enclosed by [: and :] within the enclosing square brackets. PCRE also supports -this notation. For example, -<pre> - [01[:alpha:]%] -</pre> -matches "0", "1", any alphabetic character, or "%". The supported class names -are: -<pre> - alnum letters and digits - alpha letters - ascii character codes 0 - 127 - blank space or tab only - cntrl control characters - digit decimal digits (same as \d) - graph printing characters, excluding space - lower lower case letters - print printing characters, including space - punct printing characters, excluding letters and digits and space - space white space (the same as \s from PCRE 8.34) - upper upper case letters - word "word" characters (same as \w) - xdigit hexadecimal digits -</pre> -The default "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), -and space (32). If locale-specific matching is taking place, the list of space -characters may be different; there may be fewer or more of them. "Space" used -to be different to \s, which did not include VT, for Perl compatibility. -However, Perl changed at release 5.18, and PCRE followed at release 8.34. -"Space" and \s now match the same set of characters. -</P> -<P> -The name "word" is a Perl extension, and "blank" is a GNU extension from Perl -5.8. Another Perl extension is negation, which is indicated by a ^ character -after the colon. For example, -<pre> - [12[:^digit:]] -</pre> -matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the POSIX -syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not -supported, and an error is given if they are encountered. -</P> -<P> -By default, characters with values greater than 128 do not match any of the -POSIX character classes. However, if the PCRE_UCP option is passed to -<b>pcre_compile()</b>, some of the classes are changed so that Unicode character -properties are used. This is achieved by replacing certain POSIX classes by -other sequences, as follows: -<pre> - [:alnum:] becomes \p{Xan} - [:alpha:] becomes \p{L} - [:blank:] becomes \h - [:digit:] becomes \p{Nd} - [:lower:] becomes \p{Ll} - [:space:] becomes \p{Xps} - [:upper:] becomes \p{Lu} - [:word:] becomes \p{Xwd} -</pre> -Negated versions, such as [:^alpha:] use \P instead of \p. Three other POSIX -classes are handled specially in UCP mode: -</P> -<P> -[:graph:] -This matches characters that have glyphs that mark the page when printed. In -Unicode property terms, it matches all characters with the L, M, N, P, S, or Cf -properties, except for: -<pre> - U+061C Arabic Letter Mark - U+180E Mongolian Vowel Separator - U+2066 - U+2069 Various "isolate"s - -</PRE> -</P> -<P> -[:print:] -This matches the same characters as [:graph:] plus space characters that are -not controls, that is, characters with the Zs property. -</P> -<P> -[:punct:] -This matches all characters that have the Unicode P (punctuation) property, -plus those characters whose code points are less than 128 that have the S -(Symbol) property. -</P> -<P> -The other POSIX classes are unchanged, and match only characters with code -points less than 128. -</P> -<br><a name="SEC11" href="#TOC1">COMPATIBILITY FEATURE FOR WORD BOUNDARIES</a><br> -<P> -In the POSIX.2 compliant library that was included in 4.4BSD Unix, the ugly -syntax [[:<:]] and [[:>:]] is used for matching "start of word" and "end of -word". PCRE treats these items as follows: -<pre> - [[:<:]] is converted to \b(?=\w) - [[:>:]] is converted to \b(?<=\w) -</pre> -Only these exact character sequences are recognized. A sequence such as -[a[:<:]b] provokes error for an unrecognized POSIX class name. This support is -not compatible with Perl. It is provided to help migrations from other -environments, and is best not used in any new patterns. Note that \b matches -at the start and the end of a word (see -<a href="#smallassertions">"Simple assertions"</a> -above), and in a Perl-style pattern the preceding or following character -normally shows which is wanted, without the need for the assertions that are -used above in order to give exactly the POSIX behaviour. -</P> -<br><a name="SEC12" href="#TOC1">VERTICAL BAR</a><br> -<P> -Vertical bar characters are used to separate alternative patterns. For example, -the pattern -<pre> - gilbert|sullivan -</pre> -matches either "gilbert" or "sullivan". Any number of alternatives may appear, -and an empty alternative is permitted (matching the empty string). The matching -process tries each alternative in turn, from left to right, and the first one -that succeeds is used. If the alternatives are within a subpattern -<a href="#subpattern">(defined below),</a> -"succeeds" means matching the rest of the main pattern as well as the -alternative in the subpattern. -</P> -<br><a name="SEC13" href="#TOC1">INTERNAL OPTION SETTING</a><br> -<P> -The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and -PCRE_EXTENDED options (which are Perl-compatible) can be changed from within -the pattern by a sequence of Perl option letters enclosed between "(?" and ")". -The option letters are -<pre> - i for PCRE_CASELESS - m for PCRE_MULTILINE - s for PCRE_DOTALL - x for PCRE_EXTENDED -</pre> -For example, (?im) sets caseless, multiline matching. It is also possible to -unset these options by preceding the letter with a hyphen, and a combined -setting and unsetting such as (?im-sx), which sets PCRE_CASELESS and -PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, is also -permitted. If a letter appears both before and after the hyphen, the option is -unset. -</P> -<P> -The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA can be -changed in the same way as the Perl-compatible options by using the characters -J, U and X respectively. -</P> -<P> -When one of these option changes occurs at top level (that is, not inside -subpattern parentheses), the change applies to the remainder of the pattern -that follows. If the change is placed right at the start of a pattern, PCRE -extracts it into the global options (and it will therefore show up in data -extracted by the <b>pcre_fullinfo()</b> function). -</P> -<P> -An option change within a subpattern (see below for a description of -subpatterns) affects only that part of the subpattern that follows it, so -<pre> - (a(?i)b)c -</pre> -matches abc and aBc and no other strings (assuming PCRE_CASELESS is not used). -By this means, options can be made to have different settings in different -parts of the pattern. Any changes made in one alternative do carry on -into subsequent branches within the same subpattern. For example, -<pre> - (a(?i)b|c) -</pre> -matches "ab", "aB", "c", and "C", even though when matching "C" the first -branch is abandoned before the option setting. This is because the effects of -option settings happen at compile time. There would be some very weird -behaviour otherwise. -</P> -<P> -<b>Note:</b> There are other PCRE-specific options that can be set by the -application when the compiling or matching functions are called. In some cases -the pattern can contain special leading sequences such as (*CRLF) to override -what the application has set or what has been defaulted. Details are given in -the section entitled -<a href="#newlineseq">"Newline sequences"</a> -above. There are also the (*UTF8), (*UTF16),(*UTF32), and (*UCP) leading -sequences that can be used to set UTF and Unicode property modes; they are -equivalent to setting the PCRE_UTF8, PCRE_UTF16, PCRE_UTF32 and the PCRE_UCP -options, respectively. The (*UTF) sequence is a generic version that can be -used with any of the libraries. However, the application can set the -PCRE_NEVER_UTF option, which locks out the use of the (*UTF) sequences. -<a name="subpattern"></a></P> -<br><a name="SEC14" href="#TOC1">SUBPATTERNS</a><br> -<P> -Subpatterns are delimited by parentheses (round brackets), which can be nested. -Turning part of a pattern into a subpattern does two things: -<br> -<br> -1. It localizes a set of alternatives. For example, the pattern -<pre> - cat(aract|erpillar|) -</pre> -matches "cataract", "caterpillar", or "cat". Without the parentheses, it would -match "cataract", "erpillar" or an empty string. -<br> -<br> -2. It sets up the subpattern as a capturing subpattern. This means that, when -the whole pattern matches, that portion of the subject string that matched the -subpattern is passed back to the caller via the <i>ovector</i> argument of the -matching function. (This applies only to the traditional matching functions; -the DFA matching functions do not support capturing.) -</P> -<P> -Opening parentheses are counted from left to right (starting from 1) to obtain -numbers for the capturing subpatterns. For example, if the string "the red -king" is matched against the pattern -<pre> - the ((red|white) (king|queen)) -</pre> -the captured substrings are "red king", "red", and "king", and are numbered 1, -2, and 3, respectively. -</P> -<P> -The fact that plain parentheses fulfil two functions is not always helpful. -There are often times when a grouping subpattern is required without a -capturing requirement. If an opening parenthesis is followed by a question mark -and a colon, the subpattern does not do any capturing, and is not counted when -computing the number of any subsequent capturing subpatterns. For example, if -the string "the white queen" is matched against the pattern -<pre> - the ((?:red|white) (king|queen)) -</pre> -the captured substrings are "white queen" and "queen", and are numbered 1 and -2. The maximum number of capturing subpatterns is 65535. -</P> -<P> -As a convenient shorthand, if any option settings are required at the start of -a non-capturing subpattern, the option letters may appear between the "?" and -the ":". Thus the two patterns -<pre> - (?i:saturday|sunday) - (?:(?i)saturday|sunday) -</pre> -match exactly the same set of strings. Because alternative branches are tried -from left to right, and options are not reset until the end of the subpattern -is reached, an option setting in one branch does affect subsequent branches, so -the above patterns match "SUNDAY" as well as "Saturday". -<a name="dupsubpatternnumber"></a></P> -<br><a name="SEC15" href="#TOC1">DUPLICATE SUBPATTERN NUMBERS</a><br> -<P> -Perl 5.10 introduced a feature whereby each alternative in a subpattern uses -the same numbers for its capturing parentheses. Such a subpattern starts with -(?| and is itself a non-capturing subpattern. For example, consider this -pattern: -<pre> - (?|(Sat)ur|(Sun))day -</pre> -Because the two alternatives are inside a (?| group, both sets of capturing -parentheses are numbered one. Thus, when the pattern matches, you can look -at captured substring number one, whichever alternative matched. This construct -is useful when you want to capture part, but not all, of one of a number of -alternatives. Inside a (?| group, parentheses are numbered as usual, but the -number is reset at the start of each branch. The numbers of any capturing -parentheses that follow the subpattern start after the highest number used in -any branch. The following example is taken from the Perl documentation. The -numbers underneath show in which buffer the captured content will be stored. -<pre> - # before ---------------branch-reset----------- after - / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x - # 1 2 2 3 2 3 4 -</pre> -A back reference to a numbered subpattern uses the most recent value that is -set for that number by any subpattern. The following pattern matches "abcabc" -or "defdef": -<pre> - /(?|(abc)|(def))\1/ -</pre> -In contrast, a subroutine call to a numbered subpattern always refers to the -first one in the pattern with the given number. The following pattern matches -"abcabc" or "defabc": -<pre> - /(?|(abc)|(def))(?1)/ -</pre> -If a -<a href="#conditions">condition test</a> -for a subpattern's having matched refers to a non-unique number, the test is -true if any of the subpatterns of that number have matched. -</P> -<P> -An alternative approach to using this "branch reset" feature is to use -duplicate named subpatterns, as described in the next section. -</P> -<br><a name="SEC16" href="#TOC1">NAMED SUBPATTERNS</a><br> -<P> -Identifying capturing parentheses by number is simple, but it can be very hard -to keep track of the numbers in complicated regular expressions. Furthermore, -if an expression is modified, the numbers may change. To help with this -difficulty, PCRE supports the naming of subpatterns. This feature was not -added to Perl until release 5.10. Python had the feature earlier, and PCRE -introduced it at release 4.0, using the Python syntax. PCRE now supports both -the Perl and the Python syntax. Perl allows identically numbered subpatterns to -have different names, but PCRE does not. -</P> -<P> -In PCRE, a subpattern can be named in one of three ways: (?<name>...) or -(?'name'...) as in Perl, or (?P<name>...) as in Python. References to capturing -parentheses from other parts of the pattern, such as -<a href="#backreferences">back references,</a> -<a href="#recursion">recursion,</a> -and -<a href="#conditions">conditions,</a> -can be made by name as well as by number. -</P> -<P> -Names consist of up to 32 alphanumeric characters and underscores, but must -start with a non-digit. Named capturing parentheses are still allocated numbers -as well as names, exactly as if the names were not present. The PCRE API -provides function calls for extracting the name-to-number translation table -from a compiled pattern. There is also a convenience function for extracting a -captured substring by name. -</P> -<P> -By default, a name must be unique within a pattern, but it is possible to relax -this constraint by setting the PCRE_DUPNAMES option at compile time. (Duplicate -names are also always permitted for subpatterns with the same number, set up as -described in the previous section.) Duplicate names can be useful for patterns -where only one instance of the named parentheses can match. Suppose you want to -match the name of a weekday, either as a 3-letter abbreviation or as the full -name, and in both cases you want to extract the abbreviation. This pattern -(ignoring the line breaks) does the job: -<pre> - (?<DN>Mon|Fri|Sun)(?:day)?| - (?<DN>Tue)(?:sday)?| - (?<DN>Wed)(?:nesday)?| - (?<DN>Thu)(?:rsday)?| - (?<DN>Sat)(?:urday)? -</pre> -There are five capturing substrings, but only one is ever set after a match. -(An alternative way of solving this problem is to use a "branch reset" -subpattern, as described in the previous section.) -</P> -<P> -The convenience function for extracting the data by name returns the substring -for the first (and in this example, the only) subpattern of that name that -matched. This saves searching to find which numbered subpattern it was. -</P> -<P> -If you make a back reference to a non-unique named subpattern from elsewhere in -the pattern, the subpatterns to which the name refers are checked in the order -in which they appear in the overall pattern. The first one that is set is used -for the reference. For example, this pattern matches both "foofoo" and -"barbar" but not "foobar" or "barfoo": -<pre> - (?:(?<n>foo)|(?<n>bar))\k<n> - -</PRE> -</P> -<P> -If you make a subroutine call to a non-unique named subpattern, the one that -corresponds to the first occurrence of the name is used. In the absence of -duplicate numbers (see the previous section) this is the one with the lowest -number. -</P> -<P> -If you use a named reference in a condition -test (see the -<a href="#conditions">section about conditions</a> -below), either to check whether a subpattern has matched, or to check for -recursion, all subpatterns with the same name are tested. If the condition is -true for any one of them, the overall condition is true. This is the same -behaviour as testing by number. For further details of the interfaces for -handling named subpatterns, see the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation. -</P> -<P> -<b>Warning:</b> You cannot use different names to distinguish between two -subpatterns with the same number because PCRE uses only the numbers when -matching. For this reason, an error is given at compile time if different names -are given to subpatterns with the same number. However, you can always give the -same name to subpatterns with the same number, even when PCRE_DUPNAMES is not -set. -</P> -<br><a name="SEC17" href="#TOC1">REPETITION</a><br> -<P> -Repetition is specified by quantifiers, which can follow any of the following -items: -<pre> - a literal data character - the dot metacharacter - the \C escape sequence - the \X escape sequence - the \R escape sequence - an escape such as \d or \pL that matches a single character - a character class - a back reference (see next section) - a parenthesized subpattern (including assertions) - a subroutine call to a subpattern (recursive or otherwise) -</pre> -The general repetition quantifier specifies a minimum and maximum number of -permitted matches, by giving the two numbers in curly brackets (braces), -separated by a comma. The numbers must be less than 65536, and the first must -be less than or equal to the second. For example: -<pre> - z{2,4} -</pre> -matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special -character. If the second number is omitted, but the comma is present, there is -no upper limit; if the second number and the comma are both omitted, the -quantifier specifies an exact number of required matches. Thus -<pre> - [aeiou]{3,} -</pre> -matches at least 3 successive vowels, but may match many more, while -<pre> - \d{8} -</pre> -matches exactly 8 digits. An opening curly bracket that appears in a position -where a quantifier is not allowed, or one that does not match the syntax of a -quantifier, is taken as a literal character. For example, {,6} is not a -quantifier, but a literal string of four characters. -</P> -<P> -In UTF modes, quantifiers apply to characters rather than to individual data -units. Thus, for example, \x{100}{2} matches two characters, each of -which is represented by a two-byte sequence in a UTF-8 string. Similarly, -\X{3} matches three Unicode extended grapheme clusters, each of which may be -several data units long (and they may be of different lengths). -</P> -<P> -The quantifier {0} is permitted, causing the expression to behave as if the -previous item and the quantifier were not present. This may be useful for -subpatterns that are referenced as -<a href="#subpatternsassubroutines">subroutines</a> -from elsewhere in the pattern (but see also the section entitled -<a href="#subdefine">"Defining subpatterns for use by reference only"</a> -below). Items other than subpatterns that have a {0} quantifier are omitted -from the compiled pattern. -</P> -<P> -For convenience, the three most common quantifiers have single-character -abbreviations: -<pre> - * is equivalent to {0,} - + is equivalent to {1,} - ? is equivalent to {0,1} -</pre> -It is possible to construct infinite loops by following a subpattern that can -match no characters with a quantifier that has no upper limit, for example: -<pre> - (a?)* -</pre> -Earlier versions of Perl and PCRE used to give an error at compile time for -such patterns. However, because there are cases where this can be useful, such -patterns are now accepted, but if any repetition of the subpattern does in fact -match no characters, the loop is forcibly broken. -</P> -<P> -By default, the quantifiers are "greedy", that is, they match as much as -possible (up to the maximum number of permitted times), without causing the -rest of the pattern to fail. The classic example of where this gives problems -is in trying to match comments in C programs. These appear between /* and */ -and within the comment, individual * and / characters may appear. An attempt to -match C comments by applying the pattern -<pre> - /\*.*\*/ -</pre> -to the string -<pre> - /* first comment */ not comment /* second comment */ -</pre> -fails, because it matches the entire string owing to the greediness of the .* -item. -</P> -<P> -However, if a quantifier is followed by a question mark, it ceases to be -greedy, and instead matches the minimum number of times possible, so the -pattern -<pre> - /\*.*?\*/ -</pre> -does the right thing with the C comments. The meaning of the various -quantifiers is not otherwise changed, just the preferred number of matches. -Do not confuse this use of question mark with its use as a quantifier in its -own right. Because it has two uses, it can sometimes appear doubled, as in -<pre> - \d??\d -</pre> -which matches one digit by preference, but can match two if that is the only -way the rest of the pattern matches. -</P> -<P> -If the PCRE_UNGREEDY option is set (an option that is not available in Perl), -the quantifiers are not greedy by default, but individual ones can be made -greedy by following them with a question mark. In other words, it inverts the -default behaviour. -</P> -<P> -When a parenthesized subpattern is quantified with a minimum repeat count that -is greater than 1 or with a limited maximum, more memory is required for the -compiled pattern, in proportion to the size of the minimum or maximum. -</P> -<P> -If a pattern starts with .* or .{0,} and the PCRE_DOTALL option (equivalent -to Perl's /s) is set, thus allowing the dot to match newlines, the pattern is -implicitly anchored, because whatever follows will be tried against every -character position in the subject string, so there is no point in retrying the -overall match at any position after the first. PCRE normally treats such a -pattern as though it were preceded by \A. -</P> -<P> -In cases where it is known that the subject string contains no newlines, it is -worth setting PCRE_DOTALL in order to obtain this optimization, or -alternatively using ^ to indicate anchoring explicitly. -</P> -<P> -However, there are some cases where the optimization cannot be used. When .* -is inside capturing parentheses that are the subject of a back reference -elsewhere in the pattern, a match at the start may fail where a later one -succeeds. Consider, for example: -<pre> - (.*)abc\1 -</pre> -If the subject is "xyz123abc123" the match point is the fourth character. For -this reason, such a pattern is not implicitly anchored. -</P> -<P> -Another case where implicit anchoring is not applied is when the leading .* is -inside an atomic group. Once again, a match at the start may fail where a later -one succeeds. Consider this pattern: -<pre> - (?>.*?a)b -</pre> -It matches "ab" in the subject "aab". The use of the backtracking control verbs -(*PRUNE) and (*SKIP) also disable this optimization. -</P> -<P> -When a capturing subpattern is repeated, the value captured is the substring -that matched the final iteration. For example, after -<pre> - (tweedle[dume]{3}\s*)+ -</pre> -has matched "tweedledum tweedledee" the value of the captured substring is -"tweedledee". However, if there are nested capturing subpatterns, the -corresponding captured values may have been set in previous iterations. For -example, after -<pre> - /(a|(b))+/ -</pre> -matches "aba" the value of the second captured substring is "b". -<a name="atomicgroup"></a></P> -<br><a name="SEC18" href="#TOC1">ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS</a><br> -<P> -With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") -repetition, failure of what follows normally causes the repeated item to be -re-evaluated to see if a different number of repeats allows the rest of the -pattern to match. Sometimes it is useful to prevent this, either to change the -nature of the match, or to cause it fail earlier than it otherwise might, when -the author of the pattern knows there is no point in carrying on. -</P> -<P> -Consider, for example, the pattern \d+foo when applied to the subject line -<pre> - 123456bar -</pre> -After matching all 6 digits and then failing to match "foo", the normal -action of the matcher is to try again with only 5 digits matching the \d+ -item, and then with 4, and so on, before ultimately failing. "Atomic grouping" -(a term taken from Jeffrey Friedl's book) provides the means for specifying -that once a subpattern has matched, it is not to be re-evaluated in this way. -</P> -<P> -If we use atomic grouping for the previous example, the matcher gives up -immediately on failing to match "foo" the first time. The notation is a kind of -special parenthesis, starting with (?> as in this example: -<pre> - (?>\d+)foo -</pre> -This kind of parenthesis "locks up" the part of the pattern it contains once -it has matched, and a failure further into the pattern is prevented from -backtracking into it. Backtracking past it to previous items, however, works as -normal. -</P> -<P> -An alternative description is that a subpattern of this type matches the string -of characters that an identical standalone pattern would match, if anchored at -the current point in the subject string. -</P> -<P> -Atomic grouping subpatterns are not capturing subpatterns. Simple cases such as -the above example can be thought of as a maximizing repeat that must swallow -everything it can. So, while both \d+ and \d+? are prepared to adjust the -number of digits they match in order to make the rest of the pattern match, -(?>\d+) can only match an entire sequence of digits. -</P> -<P> -Atomic groups in general can of course contain arbitrarily complicated -subpatterns, and can be nested. However, when the subpattern for an atomic -group is just a single repeated item, as in the example above, a simpler -notation, called a "possessive quantifier" can be used. This consists of an -additional + character following a quantifier. Using this notation, the -previous example can be rewritten as -<pre> - \d++foo -</pre> -Note that a possessive quantifier can be used with an entire group, for -example: -<pre> - (abc|xyz){2,3}+ -</pre> -Possessive quantifiers are always greedy; the setting of the PCRE_UNGREEDY -option is ignored. They are a convenient notation for the simpler forms of -atomic group. However, there is no difference in the meaning of a possessive -quantifier and the equivalent atomic group, though there may be a performance -difference; possessive quantifiers should be slightly faster. -</P> -<P> -The possessive quantifier syntax is an extension to the Perl 5.8 syntax. -Jeffrey Friedl originated the idea (and the name) in the first edition of his -book. Mike McCloskey liked it, so implemented it when he built Sun's Java -package, and PCRE copied it from there. It ultimately found its way into Perl -at release 5.10. -</P> -<P> -PCRE has an optimization that automatically "possessifies" certain simple -pattern constructs. For example, the sequence A+B is treated as A++B because -there is no point in backtracking into a sequence of A's when B must follow. -</P> -<P> -When a pattern contains an unlimited repeat inside a subpattern that can itself -be repeated an unlimited number of times, the use of an atomic group is the -only way to avoid some failing matches taking a very long time indeed. The -pattern -<pre> - (\D+|<\d+>)*[!?] -</pre> -matches an unlimited number of substrings that either consist of non-digits, or -digits enclosed in <>, followed by either ! or ?. When it matches, it runs -quickly. However, if it is applied to -<pre> - aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa -</pre> -it takes a long time before reporting failure. This is because the string can -be divided between the internal \D+ repeat and the external * repeat in a -large number of ways, and all have to be tried. (The example uses [!?] rather -than a single character at the end, because both PCRE and Perl have an -optimization that allows for fast failure when a single character is used. They -remember the last single character that is required for a match, and fail early -if it is not present in the string.) If the pattern is changed so that it uses -an atomic group, like this: -<pre> - ((?>\D+)|<\d+>)*[!?] -</pre> -sequences of non-digits cannot be broken, and failure happens quickly. -<a name="backreferences"></a></P> -<br><a name="SEC19" href="#TOC1">BACK REFERENCES</a><br> -<P> -Outside a character class, a backslash followed by a digit greater than 0 (and -possibly further digits) is a back reference to a capturing subpattern earlier -(that is, to its left) in the pattern, provided there have been that many -previous capturing left parentheses. -</P> -<P> -However, if the decimal number following the backslash is less than 10, it is -always taken as a back reference, and causes an error only if there are not -that many capturing left parentheses in the entire pattern. In other words, the -parentheses that are referenced need not be to the left of the reference for -numbers less than 10. A "forward back reference" of this type can make sense -when a repetition is involved and the subpattern to the right has participated -in an earlier iteration. -</P> -<P> -It is not possible to have a numerical "forward back reference" to a subpattern -whose number is 10 or more using this syntax because a sequence such as \50 is -interpreted as a character defined in octal. See the subsection entitled -"Non-printing characters" -<a href="#digitsafterbackslash">above</a> -for further details of the handling of digits following a backslash. There is -no such problem when named parentheses are used. A back reference to any -subpattern is possible using named parentheses (see below). -</P> -<P> -Another way of avoiding the ambiguity inherent in the use of digits following a -backslash is to use the \g escape sequence. This escape must be followed by an -unsigned number or a negative number, optionally enclosed in braces. These -examples are all identical: -<pre> - (ring), \1 - (ring), \g1 - (ring), \g{1} -</pre> -An unsigned number specifies an absolute reference without the ambiguity that -is present in the older syntax. It is also useful when literal digits follow -the reference. A negative number is a relative reference. Consider this -example: -<pre> - (abc(def)ghi)\g{-1} -</pre> -The sequence \g{-1} is a reference to the most recently started capturing -subpattern before \g, that is, is it equivalent to \2 in this example. -Similarly, \g{-2} would be equivalent to \1. The use of relative references -can be helpful in long patterns, and also in patterns that are created by -joining together fragments that contain references within themselves. -</P> -<P> -A back reference matches whatever actually matched the capturing subpattern in -the current subject string, rather than anything matching the subpattern -itself (see -<a href="#subpatternsassubroutines">"Subpatterns as subroutines"</a> -below for a way of doing that). So the pattern -<pre> - (sens|respons)e and \1ibility -</pre> -matches "sense and sensibility" and "response and responsibility", but not -"sense and responsibility". If caseful matching is in force at the time of the -back reference, the case of letters is relevant. For example, -<pre> - ((?i)rah)\s+\1 -</pre> -matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original -capturing subpattern is matched caselessly. -</P> -<P> -There are several different ways of writing back references to named -subpatterns. The .NET syntax \k{name} and the Perl syntax \k<name> or -\k'name' are supported, as is the Python syntax (?P=name). Perl 5.10's unified -back reference syntax, in which \g can be used for both numeric and named -references, is also supported. We could rewrite the above example in any of -the following ways: -<pre> - (?<p1>(?i)rah)\s+\k<p1> - (?'p1'(?i)rah)\s+\k{p1} - (?P<p1>(?i)rah)\s+(?P=p1) - (?<p1>(?i)rah)\s+\g{p1} -</pre> -A subpattern that is referenced by name may appear in the pattern before or -after the reference. -</P> -<P> -There may be more than one back reference to the same subpattern. If a -subpattern has not actually been used in a particular match, any back -references to it always fail by default. For example, the pattern -<pre> - (a|(bc))\2 -</pre> -always fails if it starts to match "a" rather than "bc". However, if the -PCRE_JAVASCRIPT_COMPAT option is set at compile time, a back reference to an -unset value matches an empty string. -</P> -<P> -Because there may be many capturing parentheses in a pattern, all digits -following a backslash are taken as part of a potential back reference number. -If the pattern continues with a digit character, some delimiter must be used to -terminate the back reference. If the PCRE_EXTENDED option is set, this can be -white space. Otherwise, the \g{ syntax or an empty comment (see -<a href="#comments">"Comments"</a> -below) can be used. -</P> -<br><b> -Recursive back references -</b><br> -<P> -A back reference that occurs inside the parentheses to which it refers fails -when the subpattern is first used, so, for example, (a\1) never matches. -However, such references can be useful inside repeated subpatterns. For -example, the pattern -<pre> - (a|b\1)+ -</pre> -matches any number of "a"s and also "aba", "ababbaa" etc. At each iteration of -the subpattern, the back reference matches the character string corresponding -to the previous iteration. In order for this to work, the pattern must be such -that the first iteration does not need to match the back reference. This can be -done using alternation, as in the example above, or by a quantifier with a -minimum of zero. -</P> -<P> -Back references of this type cause the group that they reference to be treated -as an -<a href="#atomicgroup">atomic group.</a> -Once the whole group has been matched, a subsequent matching failure cannot -cause backtracking into the middle of the group. -<a name="bigassertions"></a></P> -<br><a name="SEC20" href="#TOC1">ASSERTIONS</a><br> -<P> -An assertion is a test on the characters following or preceding the current -matching point that does not actually consume any characters. The simple -assertions coded as \b, \B, \A, \G, \Z, \z, ^ and $ are described -<a href="#smallassertions">above.</a> -</P> -<P> -More complicated assertions are coded as subpatterns. There are two kinds: -those that look ahead of the current position in the subject string, and those -that look behind it. An assertion subpattern is matched in the normal way, -except that it does not cause the current matching position to be changed. -</P> -<P> -Assertion subpatterns are not capturing subpatterns. If such an assertion -contains capturing subpatterns within it, these are counted for the purposes of -numbering the capturing subpatterns in the whole pattern. However, substring -capturing is carried out only for positive assertions. (Perl sometimes, but not -always, does do capturing in negative assertions.) -</P> -<P> -For compatibility with Perl, assertion subpatterns may be repeated; though -it makes no sense to assert the same thing several times, the side effect of -capturing parentheses may occasionally be useful. In practice, there only three -cases: -<br> -<br> -(1) If the quantifier is {0}, the assertion is never obeyed during matching. -However, it may contain internal capturing parenthesized groups that are called -from elsewhere via the -<a href="#subpatternsassubroutines">subroutine mechanism.</a> -<br> -<br> -(2) If quantifier is {0,n} where n is greater than zero, it is treated as if it -were {0,1}. At run time, the rest of the pattern match is tried with and -without the assertion, the order depending on the greediness of the quantifier. -<br> -<br> -(3) If the minimum repetition is greater than zero, the quantifier is ignored. -The assertion is obeyed just once when encountered during matching. -</P> -<br><b> -Lookahead assertions -</b><br> -<P> -Lookahead assertions start with (?= for positive assertions and (?! for -negative assertions. For example, -<pre> - \w+(?=;) -</pre> -matches a word followed by a semicolon, but does not include the semicolon in -the match, and -<pre> - foo(?!bar) -</pre> -matches any occurrence of "foo" that is not followed by "bar". Note that the -apparently similar pattern -<pre> - (?!foo)bar -</pre> -does not find an occurrence of "bar" that is preceded by something other than -"foo"; it finds any occurrence of "bar" whatsoever, because the assertion -(?!foo) is always true when the next three characters are "bar". A -lookbehind assertion is needed to achieve the other effect. -</P> -<P> -If you want to force a matching failure at some point in a pattern, the most -convenient way to do it is with (?!) because an empty string always matches, so -an assertion that requires there not to be an empty string must always fail. -The backtracking control verb (*FAIL) or (*F) is a synonym for (?!). -<a name="lookbehind"></a></P> -<br><b> -Lookbehind assertions -</b><br> -<P> -Lookbehind assertions start with (?<= for positive assertions and (?<! for -negative assertions. For example, -<pre> - (?<!foo)bar -</pre> -does find an occurrence of "bar" that is not preceded by "foo". The contents of -a lookbehind assertion are restricted such that all the strings it matches must -have a fixed length. However, if there are several top-level alternatives, they -do not all have to have the same fixed length. Thus -<pre> - (?<=bullock|donkey) -</pre> -is permitted, but -<pre> - (?<!dogs?|cats?) -</pre> -causes an error at compile time. Branches that match different length strings -are permitted only at the top level of a lookbehind assertion. This is an -extension compared with Perl, which requires all branches to match the same -length of string. An assertion such as -<pre> - (?<=ab(c|de)) -</pre> -is not permitted, because its single top-level branch can match two different -lengths, but it is acceptable to PCRE if rewritten to use two top-level -branches: -<pre> - (?<=abc|abde) -</pre> -In some cases, the escape sequence \K -<a href="#resetmatchstart">(see above)</a> -can be used instead of a lookbehind assertion to get round the fixed-length -restriction. -</P> -<P> -The implementation of lookbehind assertions is, for each alternative, to -temporarily move the current position back by the fixed length and then try to -match. If there are insufficient characters before the current position, the -assertion fails. -</P> -<P> -In a UTF mode, PCRE does not allow the \C escape (which matches a single data -unit even in a UTF mode) to appear in lookbehind assertions, because it makes -it impossible to calculate the length of the lookbehind. The \X and \R -escapes, which can match different numbers of data units, are also not -permitted. -</P> -<P> -<a href="#subpatternsassubroutines">"Subroutine"</a> -calls (see below) such as (?2) or (?&X) are permitted in lookbehinds, as long -as the subpattern matches a fixed-length string. -<a href="#recursion">Recursion,</a> -however, is not supported. -</P> -<P> -Possessive quantifiers can be used in conjunction with lookbehind assertions to -specify efficient matching of fixed-length strings at the end of subject -strings. Consider a simple pattern such as -<pre> - abcd$ -</pre> -when applied to a long string that does not match. Because matching proceeds -from left to right, PCRE will look for each "a" in the subject and then see if -what follows matches the rest of the pattern. If the pattern is specified as -<pre> - ^.*abcd$ -</pre> -the initial .* matches the entire string at first, but when this fails (because -there is no following "a"), it backtracks to match all but the last character, -then all but the last two characters, and so on. Once again the search for "a" -covers the entire string, from right to left, so we are no better off. However, -if the pattern is written as -<pre> - ^.*+(?<=abcd) -</pre> -there can be no backtracking for the .*+ item; it can match only the entire -string. The subsequent lookbehind assertion does a single test on the last four -characters. If it fails, the match fails immediately. For long strings, this -approach makes a significant difference to the processing time. -</P> -<br><b> -Using multiple assertions -</b><br> -<P> -Several assertions (of any sort) may occur in succession. For example, -<pre> - (?<=\d{3})(?<!999)foo -</pre> -matches "foo" preceded by three digits that are not "999". Notice that each of -the assertions is applied independently at the same point in the subject -string. First there is a check that the previous three characters are all -digits, and then there is a check that the same three characters are not "999". -This pattern does <i>not</i> match "foo" preceded by six characters, the first -of which are digits and the last three of which are not "999". For example, it -doesn't match "123abcfoo". A pattern to do that is -<pre> - (?<=\d{3}...)(?<!999)foo -</pre> -This time the first assertion looks at the preceding six characters, checking -that the first three are digits, and then the second assertion checks that the -preceding three characters are not "999". -</P> -<P> -Assertions can be nested in any combination. For example, -<pre> - (?<=(?<!foo)bar)baz -</pre> -matches an occurrence of "baz" that is preceded by "bar" which in turn is not -preceded by "foo", while -<pre> - (?<=\d{3}(?!999)...)foo -</pre> -is another pattern that matches "foo" preceded by three digits and any three -characters that are not "999". -<a name="conditions"></a></P> -<br><a name="SEC21" href="#TOC1">CONDITIONAL SUBPATTERNS</a><br> -<P> -It is possible to cause the matching process to obey a subpattern -conditionally or to choose between two alternative subpatterns, depending on -the result of an assertion, or whether a specific capturing subpattern has -already been matched. The two possible forms of conditional subpattern are: -<pre> - (?(condition)yes-pattern) - (?(condition)yes-pattern|no-pattern) -</pre> -If the condition is satisfied, the yes-pattern is used; otherwise the -no-pattern (if present) is used. If there are more than two alternatives in the -subpattern, a compile-time error occurs. Each of the two alternatives may -itself contain nested subpatterns of any form, including conditional -subpatterns; the restriction to two alternatives applies only at the level of -the condition. This pattern fragment is an example where the alternatives are -complex: -<pre> - (?(1) (A|B|C) | (D | (?(2)E|F) | E) ) - -</PRE> -</P> -<P> -There are four kinds of condition: references to subpatterns, references to -recursion, a pseudo-condition called DEFINE, and assertions. -</P> -<br><b> -Checking for a used subpattern by number -</b><br> -<P> -If the text between the parentheses consists of a sequence of digits, the -condition is true if a capturing subpattern of that number has previously -matched. If there is more than one capturing subpattern with the same number -(see the earlier -<a href="#recursion">section about duplicate subpattern numbers),</a> -the condition is true if any of them have matched. An alternative notation is -to precede the digits with a plus or minus sign. In this case, the subpattern -number is relative rather than absolute. The most recently opened parentheses -can be referenced by (?(-1), the next most recent by (?(-2), and so on. Inside -loops it can also make sense to refer to subsequent groups. The next -parentheses to be opened can be referenced as (?(+1), and so on. (The value -zero in any of these forms is not used; it provokes a compile-time error.) -</P> -<P> -Consider the following pattern, which contains non-significant white space to -make it more readable (assume the PCRE_EXTENDED option) and to divide it into -three parts for ease of discussion: -<pre> - ( \( )? [^()]+ (?(1) \) ) -</pre> -The first part matches an optional opening parenthesis, and if that -character is present, sets it as the first captured substring. The second part -matches one or more characters that are not parentheses. The third part is a -conditional subpattern that tests whether or not the first set of parentheses -matched. If they did, that is, if subject started with an opening parenthesis, -the condition is true, and so the yes-pattern is executed and a closing -parenthesis is required. Otherwise, since no-pattern is not present, the -subpattern matches nothing. In other words, this pattern matches a sequence of -non-parentheses, optionally enclosed in parentheses. -</P> -<P> -If you were embedding this pattern in a larger one, you could use a relative -reference: -<pre> - ...other stuff... ( \( )? [^()]+ (?(-1) \) ) ... -</pre> -This makes the fragment independent of the parentheses in the larger pattern. -</P> -<br><b> -Checking for a used subpattern by name -</b><br> -<P> -Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a used -subpattern by name. For compatibility with earlier versions of PCRE, which had -this facility before Perl, the syntax (?(name)...) is also recognized. -</P> -<P> -Rewriting the above example to use a named subpattern gives this: -<pre> - (?<OPEN> \( )? [^()]+ (?(<OPEN>) \) ) -</pre> -If the name used in a condition of this kind is a duplicate, the test is -applied to all subpatterns of the same name, and is true if any one of them has -matched. -</P> -<br><b> -Checking for pattern recursion -</b><br> -<P> -If the condition is the string (R), and there is no subpattern with the name R, -the condition is true if a recursive call to the whole pattern or any -subpattern has been made. If digits or a name preceded by ampersand follow the -letter R, for example: -<pre> - (?(R3)...) or (?(R&name)...) -</pre> -the condition is true if the most recent recursion is into a subpattern whose -number or name is given. This condition does not check the entire recursion -stack. If the name used in a condition of this kind is a duplicate, the test is -applied to all subpatterns of the same name, and is true if any one of them is -the most recent recursion. -</P> -<P> -At "top level", all these recursion test conditions are false. -<a href="#recursion">The syntax for recursive patterns</a> -is described below. -<a name="subdefine"></a></P> -<br><b> -Defining subpatterns for use by reference only -</b><br> -<P> -If the condition is the string (DEFINE), and there is no subpattern with the -name DEFINE, the condition is always false. In this case, there may be only one -alternative in the subpattern. It is always skipped if control reaches this -point in the pattern; the idea of DEFINE is that it can be used to define -subroutines that can be referenced from elsewhere. (The use of -<a href="#subpatternsassubroutines">subroutines</a> -is described below.) For example, a pattern to match an IPv4 address such as -"192.168.23.245" could be written like this (ignore white space and line -breaks): -<pre> - (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) ) - \b (?&byte) (\.(?&byte)){3} \b -</pre> -The first part of the pattern is a DEFINE group inside which a another group -named "byte" is defined. This matches an individual component of an IPv4 -address (a number less than 256). When matching takes place, this part of the -pattern is skipped because DEFINE acts like a false condition. The rest of the -pattern uses references to the named group to match the four dot-separated -components of an IPv4 address, insisting on a word boundary at each end. -</P> -<br><b> -Assertion conditions -</b><br> -<P> -If the condition is not in any of the above formats, it must be an assertion. -This may be a positive or negative lookahead or lookbehind assertion. Consider -this pattern, again containing non-significant white space, and with the two -alternatives on the second line: -<pre> - (?(?=[^a-z]*[a-z]) - \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) -</pre> -The condition is a positive lookahead assertion that matches an optional -sequence of non-letters followed by a letter. In other words, it tests for the -presence of at least one letter in the subject. If a letter is found, the -subject is matched against the first alternative; otherwise it is matched -against the second. This pattern matches strings in one of the two forms -dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. -<a name="comments"></a></P> -<br><a name="SEC22" href="#TOC1">COMMENTS</a><br> -<P> -There are two ways of including comments in patterns that are processed by -PCRE. In both cases, the start of the comment must not be in a character class, -nor in the middle of any other sequence of related characters such as (?: or a -subpattern name or number. The characters that make up a comment play no part -in the pattern matching. -</P> -<P> -The sequence (?# marks the start of a comment that continues up to the next -closing parenthesis. Nested parentheses are not permitted. If the PCRE_EXTENDED -option is set, an unescaped # character also introduces a comment, which in -this case continues to immediately after the next newline character or -character sequence in the pattern. Which characters are interpreted as newlines -is controlled by the options passed to a compiling function or by a special -sequence at the start of the pattern, as described in the section entitled -<a href="#newlines">"Newline conventions"</a> -above. Note that the end of this type of comment is a literal newline sequence -in the pattern; escape sequences that happen to represent a newline do not -count. For example, consider this pattern when PCRE_EXTENDED is set, and the -default newline convention is in force: -<pre> - abc #comment \n still comment -</pre> -On encountering the # character, <b>pcre_compile()</b> skips along, looking for -a newline in the pattern. The sequence \n is still literal at this stage, so -it does not terminate the comment. Only an actual character with the code value -0x0a (the default newline) does so. -<a name="recursion"></a></P> -<br><a name="SEC23" href="#TOC1">RECURSIVE PATTERNS</a><br> -<P> -Consider the problem of matching a string in parentheses, allowing for -unlimited nested parentheses. Without the use of recursion, the best that can -be done is to use a pattern that matches up to some fixed depth of nesting. It -is not possible to handle an arbitrary nesting depth. -</P> -<P> -For some time, Perl has provided a facility that allows regular expressions to -recurse (amongst other things). It does this by interpolating Perl code in the -expression at run time, and the code can refer to the expression itself. A Perl -pattern using code interpolation to solve the parentheses problem can be -created like this: -<pre> - $re = qr{\( (?: (?>[^()]+) | (?p{$re}) )* \)}x; -</pre> -The (?p{...}) item interpolates Perl code at run time, and in this case refers -recursively to the pattern in which it appears. -</P> -<P> -Obviously, PCRE cannot support the interpolation of Perl code. Instead, it -supports special syntax for recursion of the entire pattern, and also for -individual subpattern recursion. After its introduction in PCRE and Python, -this kind of recursion was subsequently introduced into Perl at release 5.10. -</P> -<P> -A special item that consists of (? followed by a number greater than zero and a -closing parenthesis is a recursive subroutine call of the subpattern of the -given number, provided that it occurs inside that subpattern. (If not, it is a -<a href="#subpatternsassubroutines">non-recursive subroutine</a> -call, which is described in the next section.) The special item (?R) or (?0) is -a recursive call of the entire regular expression. -</P> -<P> -This PCRE pattern solves the nested parentheses problem (assume the -PCRE_EXTENDED option is set so that white space is ignored): -<pre> - \( ( [^()]++ | (?R) )* \) -</pre> -First it matches an opening parenthesis. Then it matches any number of -substrings which can either be a sequence of non-parentheses, or a recursive -match of the pattern itself (that is, a correctly parenthesized substring). -Finally there is a closing parenthesis. Note the use of a possessive quantifier -to avoid backtracking into sequences of non-parentheses. -</P> -<P> -If this were part of a larger pattern, you would not want to recurse the entire -pattern, so instead you could use this: -<pre> - ( \( ( [^()]++ | (?1) )* \) ) -</pre> -We have put the pattern into parentheses, and caused the recursion to refer to -them instead of the whole pattern. -</P> -<P> -In a larger pattern, keeping track of parenthesis numbers can be tricky. This -is made easier by the use of relative references. Instead of (?1) in the -pattern above you can write (?-2) to refer to the second most recently opened -parentheses preceding the recursion. In other words, a negative number counts -capturing parentheses leftwards from the point at which it is encountered. -</P> -<P> -It is also possible to refer to subsequently opened parentheses, by writing -references such as (?+2). However, these cannot be recursive because the -reference is not inside the parentheses that are referenced. They are always -<a href="#subpatternsassubroutines">non-recursive subroutine</a> -calls, as described in the next section. -</P> -<P> -An alternative approach is to use named parentheses instead. The Perl syntax -for this is (?&name); PCRE's earlier syntax (?P>name) is also supported. We -could rewrite the above example as follows: -<pre> - (?<pn> \( ( [^()]++ | (?&pn) )* \) ) -</pre> -If there is more than one subpattern with the same name, the earliest one is -used. -</P> -<P> -This particular example pattern that we have been looking at contains nested -unlimited repeats, and so the use of a possessive quantifier for matching -strings of non-parentheses is important when applying the pattern to strings -that do not match. For example, when this pattern is applied to -<pre> - (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() -</pre> -it yields "no match" quickly. However, if a possessive quantifier is not used, -the match runs for a very long time indeed because there are so many different -ways the + and * repeats can carve up the subject, and all have to be tested -before failure can be reported. -</P> -<P> -At the end of a match, the values of capturing parentheses are those from -the outermost level. If you want to obtain intermediate values, a callout -function can be used (see below and the -<a href="pcrecallout.html"><b>pcrecallout</b></a> -documentation). If the pattern above is matched against -<pre> - (ab(cd)ef) -</pre> -the value for the inner capturing parentheses (numbered 2) is "ef", which is -the last value taken on at the top level. If a capturing subpattern is not -matched at the top level, its final captured value is unset, even if it was -(temporarily) set at a deeper level during the matching process. -</P> -<P> -If there are more than 15 capturing parentheses in a pattern, PCRE has to -obtain extra memory to store data during a recursion, which it does by using -<b>pcre_malloc</b>, freeing it via <b>pcre_free</b> afterwards. If no memory can -be obtained, the match fails with the PCRE_ERROR_NOMEMORY error. -</P> -<P> -Do not confuse the (?R) item with the condition (R), which tests for recursion. -Consider this pattern, which matches text in angle brackets, allowing for -arbitrary nesting. Only digits are allowed in nested brackets (that is, when -recursing), whereas any characters are permitted at the outer level. -<pre> - < (?: (?(R) \d++ | [^<>]*+) | (?R)) * > -</pre> -In this pattern, (?(R) is the start of a conditional subpattern, with two -different alternatives for the recursive and non-recursive cases. The (?R) item -is the actual recursive call. -<a name="recursiondifference"></a></P> -<br><b> -Differences in recursion processing between PCRE and Perl -</b><br> -<P> -Recursion processing in PCRE differs from Perl in two important ways. In PCRE -(like Python, but unlike Perl), a recursive subpattern call is always treated -as an atomic group. That is, once it has matched some of the subject string, it -is never re-entered, even if it contains untried alternatives and there is a -subsequent matching failure. This can be illustrated by the following pattern, -which purports to match a palindromic string that contains an odd number of -characters (for example, "a", "aba", "abcba", "abcdcba"): -<pre> - ^(.|(.)(?1)\2)$ -</pre> -The idea is that it either matches a single character, or two identical -characters surrounding a sub-palindrome. In Perl, this pattern works; in PCRE -it does not if the pattern is longer than three characters. Consider the -subject string "abcba": -</P> -<P> -At the top level, the first character is matched, but as it is not at the end -of the string, the first alternative fails; the second alternative is taken -and the recursion kicks in. The recursive call to subpattern 1 successfully -matches the next character ("b"). (Note that the beginning and end of line -tests are not part of the recursion). -</P> -<P> -Back at the top level, the next character ("c") is compared with what -subpattern 2 matched, which was "a". This fails. Because the recursion is -treated as an atomic group, there are now no backtracking points, and so the -entire match fails. (Perl is able, at this point, to re-enter the recursion and -try the second alternative.) However, if the pattern is written with the -alternatives in the other order, things are different: -<pre> - ^((.)(?1)\2|.)$ -</pre> -This time, the recursing alternative is tried first, and continues to recurse -until it runs out of characters, at which point the recursion fails. But this -time we do have another alternative to try at the higher level. That is the big -difference: in the previous case the remaining alternative is at a deeper -recursion level, which PCRE cannot use. -</P> -<P> -To change the pattern so that it matches all palindromic strings, not just -those with an odd number of characters, it is tempting to change the pattern to -this: -<pre> - ^((.)(?1)\2|.?)$ -</pre> -Again, this works in Perl, but not in PCRE, and for the same reason. When a -deeper recursion has matched a single character, it cannot be entered again in -order to match an empty string. The solution is to separate the two cases, and -write out the odd and even cases as alternatives at the higher level: -<pre> - ^(?:((.)(?1)\2|)|((.)(?3)\4|.)) -</pre> -If you want to match typical palindromic phrases, the pattern has to ignore all -non-word characters, which can be done like this: -<pre> - ^\W*+(?:((.)\W*+(?1)\W*+\2|)|((.)\W*+(?3)\W*+\4|\W*+.\W*+))\W*+$ -</pre> -If run with the PCRE_CASELESS option, this pattern matches phrases such as "A -man, a plan, a canal: Panama!" and it works well in both PCRE and Perl. Note -the use of the possessive quantifier *+ to avoid backtracking into sequences of -non-word characters. Without this, PCRE takes a great deal longer (ten times or -more) to match typical phrases, and Perl takes so long that you think it has -gone into a loop. -</P> -<P> -<b>WARNING</b>: The palindrome-matching patterns above work only if the subject -string does not start with a palindrome that is shorter than the entire string. -For example, although "abcba" is correctly matched, if the subject is "ababa", -PCRE finds the palindrome "aba" at the start, then fails at top level because -the end of the string does not follow. Once again, it cannot jump back into the -recursion to try other alternatives, so the entire match fails. -</P> -<P> -The second way in which PCRE and Perl differ in their recursion processing is -in the handling of captured values. In Perl, when a subpattern is called -recursively or as a subpattern (see the next section), it has no access to any -values that were captured outside the recursion, whereas in PCRE these values -can be referenced. Consider this pattern: -<pre> - ^(.)(\1|a(?2)) -</pre> -In PCRE, this pattern matches "bab". The first capturing parentheses match "b", -then in the second group, when the back reference \1 fails to match "b", the -second alternative matches "a" and then recurses. In the recursion, \1 does -now match "b" and so the whole match succeeds. In Perl, the pattern fails to -match because inside the recursive call \1 cannot access the externally set -value. -<a name="subpatternsassubroutines"></a></P> -<br><a name="SEC24" href="#TOC1">SUBPATTERNS AS SUBROUTINES</a><br> -<P> -If the syntax for a recursive subpattern call (either by number or by -name) is used outside the parentheses to which it refers, it operates like a -subroutine in a programming language. The called subpattern may be defined -before or after the reference. A numbered reference can be absolute or -relative, as in these examples: -<pre> - (...(absolute)...)...(?2)... - (...(relative)...)...(?-1)... - (...(?+1)...(relative)... -</pre> -An earlier example pointed out that the pattern -<pre> - (sens|respons)e and \1ibility -</pre> -matches "sense and sensibility" and "response and responsibility", but not -"sense and responsibility". If instead the pattern -<pre> - (sens|respons)e and (?1)ibility -</pre> -is used, it does match "sense and responsibility" as well as the other two -strings. Another example is given in the discussion of DEFINE above. -</P> -<P> -All subroutine calls, whether recursive or not, are always treated as atomic -groups. That is, once a subroutine has matched some of the subject string, it -is never re-entered, even if it contains untried alternatives and there is a -subsequent matching failure. Any capturing parentheses that are set during the -subroutine call revert to their previous values afterwards. -</P> -<P> -Processing options such as case-independence are fixed when a subpattern is -defined, so if it is used as a subroutine, such options cannot be changed for -different calls. For example, consider this pattern: -<pre> - (abc)(?i:(?-1)) -</pre> -It matches "abcabc". It does not match "abcABC" because the change of -processing option does not affect the called subpattern. -<a name="onigurumasubroutines"></a></P> -<br><a name="SEC25" href="#TOC1">ONIGURUMA SUBROUTINE SYNTAX</a><br> -<P> -For compatibility with Oniguruma, the non-Perl syntax \g followed by a name or -a number enclosed either in angle brackets or single quotes, is an alternative -syntax for referencing a subpattern as a subroutine, possibly recursively. Here -are two of the examples used above, rewritten using this syntax: -<pre> - (?<pn> \( ( (?>[^()]+) | \g<pn> )* \) ) - (sens|respons)e and \g'1'ibility -</pre> -PCRE supports an extension to Oniguruma: if a number is preceded by a -plus or a minus sign it is taken as a relative reference. For example: -<pre> - (abc)(?i:\g<-1>) -</pre> -Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are <i>not</i> -synonymous. The former is a back reference; the latter is a subroutine call. -</P> -<br><a name="SEC26" href="#TOC1">CALLOUTS</a><br> -<P> -Perl has a feature whereby using the sequence (?{...}) causes arbitrary Perl -code to be obeyed in the middle of matching a regular expression. This makes it -possible, amongst other things, to extract different substrings that match the -same pair of parentheses when there is a repetition. -</P> -<P> -PCRE provides a similar feature, but of course it cannot obey arbitrary Perl -code. The feature is called "callout". The caller of PCRE provides an external -function by putting its entry point in the global variable <i>pcre_callout</i> -(8-bit library) or <i>pcre[16|32]_callout</i> (16-bit or 32-bit library). -By default, this variable contains NULL, which disables all calling out. -</P> -<P> -Within a regular expression, (?C) indicates the points at which the external -function is to be called. If you want to identify different callout points, you -can put a number less than 256 after the letter C. The default value is zero. -For example, this pattern has two callout points: -<pre> - (?C1)abc(?C2)def -</pre> -If the PCRE_AUTO_CALLOUT flag is passed to a compiling function, callouts are -automatically installed before each item in the pattern. They are all numbered -255. If there is a conditional group in the pattern whose condition is an -assertion, an additional callout is inserted just before the condition. An -explicit callout may also be set at this position, as in this example: -<pre> - (?(?C9)(?=a)abc|def) -</pre> -Note that this applies only to assertion conditions, not to other types of -condition. -</P> -<P> -During matching, when PCRE reaches a callout point, the external function is -called. It is provided with the number of the callout, the position in the -pattern, and, optionally, one item of data originally supplied by the caller of -the matching function. The callout function may cause matching to proceed, to -backtrack, or to fail altogether. -</P> -<P> -By default, PCRE implements a number of optimizations at compile time and -matching time, and one side-effect is that sometimes callouts are skipped. If -you need all possible callouts to happen, you need to set options that disable -the relevant optimizations. More details, and a complete description of the -interface to the callout function, are given in the -<a href="pcrecallout.html"><b>pcrecallout</b></a> -documentation. -<a name="backtrackcontrol"></a></P> -<br><a name="SEC27" href="#TOC1">BACKTRACKING CONTROL</a><br> -<P> -Perl 5.10 introduced a number of "Special Backtracking Control Verbs", which -are still described in the Perl documentation as "experimental and subject to -change or removal in a future version of Perl". It goes on to say: "Their usage -in production code should be noted to avoid problems during upgrades." The same -remarks apply to the PCRE features described in this section. -</P> -<P> -The new verbs make use of what was previously invalid syntax: an opening -parenthesis followed by an asterisk. They are generally of the form -(*VERB) or (*VERB:NAME). Some may take either form, possibly behaving -differently depending on whether or not a name is present. A name is any -sequence of characters that does not include a closing parenthesis. The maximum -length of name is 255 in the 8-bit library and 65535 in the 16-bit and 32-bit -libraries. If the name is empty, that is, if the closing parenthesis -immediately follows the colon, the effect is as if the colon were not there. -Any number of these verbs may occur in a pattern. -</P> -<P> -Since these verbs are specifically related to backtracking, most of them can be -used only when the pattern is to be matched using one of the traditional -matching functions, because these use a backtracking algorithm. With the -exception of (*FAIL), which behaves like a failing negative assertion, the -backtracking control verbs cause an error if encountered by a DFA matching -function. -</P> -<P> -The behaviour of these verbs in -<a href="#btrepeat">repeated groups,</a> -<a href="#btassert">assertions,</a> -and in -<a href="#btsub">subpatterns called as subroutines</a> -(whether or not recursively) is documented below. -<a name="nooptimize"></a></P> -<br><b> -Optimizations that affect backtracking verbs -</b><br> -<P> -PCRE contains some optimizations that are used to speed up matching by running -some checks at the start of each match attempt. For example, it may know the -minimum length of matching subject, or that a particular character must be -present. When one of these optimizations bypasses the running of a match, any -included backtracking verbs will not, of course, be processed. You can suppress -the start-of-match optimizations by setting the PCRE_NO_START_OPTIMIZE option -when calling <b>pcre_compile()</b> or <b>pcre_exec()</b>, or by starting the -pattern with (*NO_START_OPT). There is more discussion of this option in the -section entitled -<a href="pcreapi.html#execoptions">"Option bits for <b>pcre_exec()</b>"</a> -in the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation. -</P> -<P> -Experiments with Perl suggest that it too has similar optimizations, sometimes -leading to anomalous results. -</P> -<br><b> -Verbs that act immediately -</b><br> -<P> -The following verbs act as soon as they are encountered. They may not be -followed by a name. -<pre> - (*ACCEPT) -</pre> -This verb causes the match to end successfully, skipping the remainder of the -pattern. However, when it is inside a subpattern that is called as a -subroutine, only that subpattern is ended successfully. Matching then continues -at the outer level. If (*ACCEPT) in triggered in a positive assertion, the -assertion succeeds; in a negative assertion, the assertion fails. -</P> -<P> -If (*ACCEPT) is inside capturing parentheses, the data so far is captured. For -example: -<pre> - A((?:A|B(*ACCEPT)|C)D) -</pre> -This matches "AB", "AAD", or "ACD"; when it matches "AB", "B" is captured by -the outer parentheses. -<pre> - (*FAIL) or (*F) -</pre> -This verb causes a matching failure, forcing backtracking to occur. It is -equivalent to (?!) but easier to read. The Perl documentation notes that it is -probably useful only when combined with (?{}) or (??{}). Those are, of course, -Perl features that are not present in PCRE. The nearest equivalent is the -callout feature, as for example in this pattern: -<pre> - a+(?C)(*FAIL) -</pre> -A match with the string "aaaa" always fails, but the callout is taken before -each backtrack happens (in this example, 10 times). -</P> -<br><b> -Recording which path was taken -</b><br> -<P> -There is one verb whose main purpose is to track how a match was arrived at, -though it also has a secondary use in conjunction with advancing the match -starting point (see (*SKIP) below). -<pre> - (*MARK:NAME) or (*:NAME) -</pre> -A name is always required with this verb. There may be as many instances of -(*MARK) as you like in a pattern, and their names do not have to be unique. -</P> -<P> -When a match succeeds, the name of the last-encountered (*MARK:NAME), -(*PRUNE:NAME), or (*THEN:NAME) on the matching path is passed back to the -caller as described in the section entitled -<a href="pcreapi.html#extradata">"Extra data for <b>pcre_exec()</b>"</a> -in the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation. Here is an example of <b>pcretest</b> output, where the /K -modifier requests the retrieval and outputting of (*MARK) data: -<pre> - re> /X(*MARK:A)Y|X(*MARK:B)Z/K - data> XY - 0: XY - MK: A - XZ - 0: XZ - MK: B -</pre> -The (*MARK) name is tagged with "MK:" in this output, and in this example it -indicates which of the two alternatives matched. This is a more efficient way -of obtaining this information than putting each alternative in its own -capturing parentheses. -</P> -<P> -If a verb with a name is encountered in a positive assertion that is true, the -name is recorded and passed back if it is the last-encountered. This does not -happen for negative assertions or failing positive assertions. -</P> -<P> -After a partial match or a failed match, the last encountered name in the -entire match process is returned. For example: -<pre> - re> /X(*MARK:A)Y|X(*MARK:B)Z/K - data> XP - No match, mark = B -</pre> -Note that in this unanchored example the mark is retained from the match -attempt that started at the letter "X" in the subject. Subsequent match -attempts starting at "P" and then with an empty string do not get as far as the -(*MARK) item, but nevertheless do not reset it. -</P> -<P> -If you are interested in (*MARK) values after failed matches, you should -probably set the PCRE_NO_START_OPTIMIZE option -<a href="#nooptimize">(see above)</a> -to ensure that the match is always attempted. -</P> -<br><b> -Verbs that act after backtracking -</b><br> -<P> -The following verbs do nothing when they are encountered. Matching continues -with what follows, but if there is no subsequent match, causing a backtrack to -the verb, a failure is forced. That is, backtracking cannot pass to the left of -the verb. However, when one of these verbs appears inside an atomic group or an -assertion that is true, its effect is confined to that group, because once the -group has been matched, there is never any backtracking into it. In this -situation, backtracking can "jump back" to the left of the entire atomic group -or assertion. (Remember also, as stated above, that this localization also -applies in subroutine calls.) -</P> -<P> -These verbs differ in exactly what kind of failure occurs when backtracking -reaches them. The behaviour described below is what happens when the verb is -not in a subroutine or an assertion. Subsequent sections cover these special -cases. -<pre> - (*COMMIT) -</pre> -This verb, which may not be followed by a name, causes the whole match to fail -outright if there is a later matching failure that causes backtracking to reach -it. Even if the pattern is unanchored, no further attempts to find a match by -advancing the starting point take place. If (*COMMIT) is the only backtracking -verb that is encountered, once it has been passed <b>pcre_exec()</b> is -committed to finding a match at the current starting point, or not at all. For -example: -<pre> - a+(*COMMIT)b -</pre> -This matches "xxaab" but not "aacaab". It can be thought of as a kind of -dynamic anchor, or "I've started, so I must finish." The name of the most -recently passed (*MARK) in the path is passed back when (*COMMIT) forces a -match failure. -</P> -<P> -If there is more than one backtracking verb in a pattern, a different one that -follows (*COMMIT) may be triggered first, so merely passing (*COMMIT) during a -match does not always guarantee that a match must be at this starting point. -</P> -<P> -Note that (*COMMIT) at the start of a pattern is not the same as an anchor, -unless PCRE's start-of-match optimizations are turned off, as shown in this -output from <b>pcretest</b>: -<pre> - re> /(*COMMIT)abc/ - data> xyzabc - 0: abc - data> xyzabc\Y - No match -</pre> -For this pattern, PCRE knows that any match must start with "a", so the -optimization skips along the subject to "a" before applying the pattern to the -first set of data. The match attempt then succeeds. In the second set of data, -the escape sequence \Y is interpreted by the <b>pcretest</b> program. It causes -the PCRE_NO_START_OPTIMIZE option to be set when <b>pcre_exec()</b> is called. -This disables the optimization that skips along to the first character. The -pattern is now applied starting at "x", and so the (*COMMIT) causes the match -to fail without trying any other starting points. -<pre> - (*PRUNE) or (*PRUNE:NAME) -</pre> -This verb causes the match to fail at the current starting position in the -subject if there is a later matching failure that causes backtracking to reach -it. If the pattern is unanchored, the normal "bumpalong" advance to the next -starting character then happens. Backtracking can occur as usual to the left of -(*PRUNE), before it is reached, or when matching to the right of (*PRUNE), but -if there is no match to the right, backtracking cannot cross (*PRUNE). In -simple cases, the use of (*PRUNE) is just an alternative to an atomic group or -possessive quantifier, but there are some uses of (*PRUNE) that cannot be -expressed in any other way. In an anchored pattern (*PRUNE) has the same effect -as (*COMMIT). -</P> -<P> -The behaviour of (*PRUNE:NAME) is the not the same as (*MARK:NAME)(*PRUNE). -It is like (*MARK:NAME) in that the name is remembered for passing back to the -caller. However, (*SKIP:NAME) searches only for names set with (*MARK). -<pre> - (*SKIP) -</pre> -This verb, when given without a name, is like (*PRUNE), except that if the -pattern is unanchored, the "bumpalong" advance is not to the next character, -but to the position in the subject where (*SKIP) was encountered. (*SKIP) -signifies that whatever text was matched leading up to it cannot be part of a -successful match. Consider: -<pre> - a+(*SKIP)b -</pre> -If the subject is "aaaac...", after the first match attempt fails (starting at -the first character in the string), the starting point skips on to start the -next attempt at "c". Note that a possessive quantifer does not have the same -effect as this example; although it would suppress backtracking during the -first match attempt, the second attempt would start at the second character -instead of skipping on to "c". -<pre> - (*SKIP:NAME) -</pre> -When (*SKIP) has an associated name, its behaviour is modified. When it is -triggered, the previous path through the pattern is searched for the most -recent (*MARK) that has the same name. If one is found, the "bumpalong" advance -is to the subject position that corresponds to that (*MARK) instead of to where -(*SKIP) was encountered. If no (*MARK) with a matching name is found, the -(*SKIP) is ignored. -</P> -<P> -Note that (*SKIP:NAME) searches only for names set by (*MARK:NAME). It ignores -names that are set by (*PRUNE:NAME) or (*THEN:NAME). -<pre> - (*THEN) or (*THEN:NAME) -</pre> -This verb causes a skip to the next innermost alternative when backtracking -reaches it. That is, it cancels any further backtracking within the current -alternative. Its name comes from the observation that it can be used for a -pattern-based if-then-else block: -<pre> - ( COND1 (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ ) ... -</pre> -If the COND1 pattern matches, FOO is tried (and possibly further items after -the end of the group if FOO succeeds); on failure, the matcher skips to the -second alternative and tries COND2, without backtracking into COND1. If that -succeeds and BAR fails, COND3 is tried. If subsequently BAZ fails, there are no -more alternatives, so there is a backtrack to whatever came before the entire -group. If (*THEN) is not inside an alternation, it acts like (*PRUNE). -</P> -<P> -The behaviour of (*THEN:NAME) is the not the same as (*MARK:NAME)(*THEN). -It is like (*MARK:NAME) in that the name is remembered for passing back to the -caller. However, (*SKIP:NAME) searches only for names set with (*MARK). -</P> -<P> -A subpattern that does not contain a | character is just a part of the -enclosing alternative; it is not a nested alternation with only one -alternative. The effect of (*THEN) extends beyond such a subpattern to the -enclosing alternative. Consider this pattern, where A, B, etc. are complex -pattern fragments that do not contain any | characters at this level: -<pre> - A (B(*THEN)C) | D -</pre> -If A and B are matched, but there is a failure in C, matching does not -backtrack into A; instead it moves to the next alternative, that is, D. -However, if the subpattern containing (*THEN) is given an alternative, it -behaves differently: -<pre> - A (B(*THEN)C | (*FAIL)) | D -</pre> -The effect of (*THEN) is now confined to the inner subpattern. After a failure -in C, matching moves to (*FAIL), which causes the whole subpattern to fail -because there are no more alternatives to try. In this case, matching does now -backtrack into A. -</P> -<P> -Note that a conditional subpattern is not considered as having two -alternatives, because only one is ever used. In other words, the | character in -a conditional subpattern has a different meaning. Ignoring white space, -consider: -<pre> - ^.*? (?(?=a) a | b(*THEN)c ) -</pre> -If the subject is "ba", this pattern does not match. Because .*? is ungreedy, -it initially matches zero characters. The condition (?=a) then fails, the -character "b" is matched, but "c" is not. At this point, matching does not -backtrack to .*? as might perhaps be expected from the presence of the | -character. The conditional subpattern is part of the single alternative that -comprises the whole pattern, and so the match fails. (If there was a backtrack -into .*?, allowing it to match "b", the match would succeed.) -</P> -<P> -The verbs just described provide four different "strengths" of control when -subsequent matching fails. (*THEN) is the weakest, carrying on the match at the -next alternative. (*PRUNE) comes next, failing the match at the current -starting position, but allowing an advance to the next character (for an -unanchored pattern). (*SKIP) is similar, except that the advance may be more -than one character. (*COMMIT) is the strongest, causing the entire match to -fail. -</P> -<br><b> -More than one backtracking verb -</b><br> -<P> -If more than one backtracking verb is present in a pattern, the one that is -backtracked onto first acts. For example, consider this pattern, where A, B, -etc. are complex pattern fragments: -<pre> - (A(*COMMIT)B(*THEN)C|ABD) -</pre> -If A matches but B fails, the backtrack to (*COMMIT) causes the entire match to -fail. However, if A and B match, but C fails, the backtrack to (*THEN) causes -the next alternative (ABD) to be tried. This behaviour is consistent, but is -not always the same as Perl's. It means that if two or more backtracking verbs -appear in succession, all the the last of them has no effect. Consider this -example: -<pre> - ...(*COMMIT)(*PRUNE)... -</pre> -If there is a matching failure to the right, backtracking onto (*PRUNE) causes -it to be triggered, and its action is taken. There can never be a backtrack -onto (*COMMIT). -<a name="btrepeat"></a></P> -<br><b> -Backtracking verbs in repeated groups -</b><br> -<P> -PCRE differs from Perl in its handling of backtracking verbs in repeated -groups. For example, consider: -<pre> - /(a(*COMMIT)b)+ac/ -</pre> -If the subject is "abac", Perl matches, but PCRE fails because the (*COMMIT) in -the second repeat of the group acts. -<a name="btassert"></a></P> -<br><b> -Backtracking verbs in assertions -</b><br> -<P> -(*FAIL) in an assertion has its normal effect: it forces an immediate backtrack. -</P> -<P> -(*ACCEPT) in a positive assertion causes the assertion to succeed without any -further processing. In a negative assertion, (*ACCEPT) causes the assertion to -fail without any further processing. -</P> -<P> -The other backtracking verbs are not treated specially if they appear in a -positive assertion. In particular, (*THEN) skips to the next alternative in the -innermost enclosing group that has alternations, whether or not this is within -the assertion. -</P> -<P> -Negative assertions are, however, different, in order to ensure that changing a -positive assertion into a negative assertion changes its result. Backtracking -into (*COMMIT), (*SKIP), or (*PRUNE) causes a negative assertion to be true, -without considering any further alternative branches in the assertion. -Backtracking into (*THEN) causes it to skip to the next enclosing alternative -within the assertion (the normal behaviour), but if the assertion does not have -such an alternative, (*THEN) behaves like (*PRUNE). -<a name="btsub"></a></P> -<br><b> -Backtracking verbs in subroutines -</b><br> -<P> -These behaviours occur whether or not the subpattern is called recursively. -Perl's treatment of subroutines is different in some cases. -</P> -<P> -(*FAIL) in a subpattern called as a subroutine has its normal effect: it forces -an immediate backtrack. -</P> -<P> -(*ACCEPT) in a subpattern called as a subroutine causes the subroutine match to -succeed without any further processing. Matching then continues after the -subroutine call. -</P> -<P> -(*COMMIT), (*SKIP), and (*PRUNE) in a subpattern called as a subroutine cause -the subroutine match to fail. -</P> -<P> -(*THEN) skips to the next alternative in the innermost enclosing group within -the subpattern that has alternatives. If there is no such group within the -subpattern, (*THEN) causes the subroutine match to fail. -</P> -<br><a name="SEC28" href="#TOC1">SEE ALSO</a><br> -<P> -<b>pcreapi</b>(3), <b>pcrecallout</b>(3), <b>pcrematching</b>(3), -<b>pcresyntax</b>(3), <b>pcre</b>(3), <b>pcre16(3)</b>, <b>pcre32(3)</b>. -</P> -<br><a name="SEC29" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC30" href="#TOC1">REVISION</a><br> -<P> -Last updated: 08 January 2014 -<br> -Copyright © 1997-2014 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcreperform.html b/plugins/Pcre16/docs/doc/html/pcreperform.html deleted file mode 100644 index dda207f901..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcreperform.html +++ /dev/null @@ -1,195 +0,0 @@ -<html> -<head> -<title>pcreperform specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcreperform man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -PCRE PERFORMANCE -</b><br> -<P> -Two aspects of performance are discussed below: memory usage and processing -time. The way you express your pattern as a regular expression can affect both -of them. -</P> -<br><b> -COMPILED PATTERN MEMORY USAGE -</b><br> -<P> -Patterns are compiled by PCRE into a reasonably efficient interpretive code, so -that most simple patterns do not use much memory. However, there is one case -where the memory usage of a compiled pattern can be unexpectedly large. If a -parenthesized subpattern has a quantifier with a minimum greater than 1 and/or -a limited maximum, the whole subpattern is repeated in the compiled code. For -example, the pattern -<pre> - (abc|def){2,4} -</pre> -is compiled as if it were -<pre> - (abc|def)(abc|def)((abc|def)(abc|def)?)? -</pre> -(Technical aside: It is done this way so that backtrack points within each of -the repetitions can be independently maintained.) -</P> -<P> -For regular expressions whose quantifiers use only small numbers, this is not -usually a problem. However, if the numbers are large, and particularly if such -repetitions are nested, the memory usage can become an embarrassment. For -example, the very simple pattern -<pre> - ((ab){1,1000}c){1,3} -</pre> -uses 51K bytes when compiled using the 8-bit library. When PCRE is compiled -with its default internal pointer size of two bytes, the size limit on a -compiled pattern is 64K data units, and this is reached with the above pattern -if the outer repetition is increased from 3 to 4. PCRE can be compiled to use -larger internal pointers and thus handle larger compiled patterns, but it is -better to try to rewrite your pattern to use less memory if you can. -</P> -<P> -One way of reducing the memory usage for such patterns is to make use of PCRE's -<a href="pcrepattern.html#subpatternsassubroutines">"subroutine"</a> -facility. Re-writing the above pattern as -<pre> - ((ab)(?2){0,999}c)(?1){0,2} -</pre> -reduces the memory requirements to 18K, and indeed it remains under 20K even -with the outer repetition increased to 100. However, this pattern is not -exactly equivalent, because the "subroutine" calls are treated as -<a href="pcrepattern.html#atomicgroup">atomic groups</a> -into which there can be no backtracking if there is a subsequent matching -failure. Therefore, PCRE cannot do this kind of rewriting automatically. -Furthermore, there is a noticeable loss of speed when executing the modified -pattern. Nevertheless, if the atomic grouping is not a problem and the loss of -speed is acceptable, this kind of rewriting will allow you to process patterns -that PCRE cannot otherwise handle. -</P> -<br><b> -STACK USAGE AT RUN TIME -</b><br> -<P> -When <b>pcre_exec()</b> or <b>pcre[16|32]_exec()</b> is used for matching, certain -kinds of pattern can cause it to use large amounts of the process stack. In -some environments the default process stack is quite small, and if it runs out -the result is often SIGSEGV. This issue is probably the most frequently raised -problem with PCRE. Rewriting your pattern can often help. The -<a href="pcrestack.html"><b>pcrestack</b></a> -documentation discusses this issue in detail. -</P> -<br><b> -PROCESSING TIME -</b><br> -<P> -Certain items in regular expression patterns are processed more efficiently -than others. It is more efficient to use a character class like [aeiou] than a -set of single-character alternatives such as (a|e|i|o|u). In general, the -simplest construction that provides the required behaviour is usually the most -efficient. Jeffrey Friedl's book contains a lot of useful general discussion -about optimizing regular expressions for efficient performance. This document -contains a few observations about PCRE. -</P> -<P> -Using Unicode character properties (the \p, \P, and \X escapes) is slow, -because PCRE has to use a multi-stage table lookup whenever it needs a -character's property. If you can find an alternative pattern that does not use -character properties, it will probably be faster. -</P> -<P> -By default, the escape sequences \b, \d, \s, and \w, and the POSIX -character classes such as [:alpha:] do not use Unicode properties, partly for -backwards compatibility, and partly for performance reasons. However, you can -set PCRE_UCP if you want Unicode character properties to be used. This can -double the matching time for items such as \d, when matched with -a traditional matching function; the performance loss is less with -a DFA matching function, and in both cases there is not much difference for -\b. -</P> -<P> -When a pattern begins with .* not in parentheses, or in parentheses that are -not the subject of a backreference, and the PCRE_DOTALL option is set, the -pattern is implicitly anchored by PCRE, since it can match only at the start of -a subject string. However, if PCRE_DOTALL is not set, PCRE cannot make this -optimization, because the . metacharacter does not then match a newline, and if -the subject string contains newlines, the pattern may match from the character -immediately following one of them instead of from the very start. For example, -the pattern -<pre> - .*second -</pre> -matches the subject "first\nand second" (where \n stands for a newline -character), with the match starting at the seventh character. In order to do -this, PCRE has to retry the match starting after every newline in the subject. -</P> -<P> -If you are using such a pattern with subject strings that do not contain -newlines, the best performance is obtained by setting PCRE_DOTALL, or starting -the pattern with ^.* or ^.*? to indicate explicit anchoring. That saves PCRE -from having to scan along the subject looking for a newline to restart at. -</P> -<P> -Beware of patterns that contain nested indefinite repeats. These can take a -long time to run when applied to a string that does not match. Consider the -pattern fragment -<pre> - ^(a+)* -</pre> -This can match "aaaa" in 16 different ways, and this number increases very -rapidly as the string gets longer. (The * repeat can match 0, 1, 2, 3, or 4 -times, and for each of those cases other than 0 or 4, the + repeats can match -different numbers of times.) When the remainder of the pattern is such that the -entire match is going to fail, PCRE has in principle to try every possible -variation, and this can take an extremely long time, even for relatively short -strings. -</P> -<P> -An optimization catches some of the more simple cases such as -<pre> - (a+)*b -</pre> -where a literal character follows. Before embarking on the standard matching -procedure, PCRE checks that there is a "b" later in the subject string, and if -there is not, it fails the match immediately. However, when there is no -following literal this optimization cannot be used. You can see the difference -by comparing the behaviour of -<pre> - (a+)*\d -</pre> -with the pattern above. The former gives a failure almost instantly when -applied to a whole line of "a" characters, whereas the latter takes an -appreciable time with strings longer than about 20 characters. -</P> -<P> -In many cases, the solution to this kind of performance issue is to use an -atomic group or a possessive quantifier. -</P> -<br><b> -AUTHOR -</b><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><b> -REVISION -</b><br> -<P> -Last updated: 25 August 2012 -<br> -Copyright © 1997-2012 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcreposix.html b/plugins/Pcre16/docs/doc/html/pcreposix.html deleted file mode 100644 index 18924cf7f9..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcreposix.html +++ /dev/null @@ -1,290 +0,0 @@ -<html> -<head> -<title>pcreposix specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcreposix man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">SYNOPSIS</a> -<li><a name="TOC2" href="#SEC2">DESCRIPTION</a> -<li><a name="TOC3" href="#SEC3">COMPILING A PATTERN</a> -<li><a name="TOC4" href="#SEC4">MATCHING NEWLINE CHARACTERS</a> -<li><a name="TOC5" href="#SEC5">MATCHING A PATTERN</a> -<li><a name="TOC6" href="#SEC6">ERROR MESSAGES</a> -<li><a name="TOC7" href="#SEC7">MEMORY USAGE</a> -<li><a name="TOC8" href="#SEC8">AUTHOR</a> -<li><a name="TOC9" href="#SEC9">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">SYNOPSIS</a><br> -<P> -<b>#include <pcreposix.h></b> -</P> -<P> -<b>int regcomp(regex_t *<i>preg</i>, const char *<i>pattern</i>,</b> -<b> int <i>cflags</i>);</b> -<br> -<br> -<b>int regexec(regex_t *<i>preg</i>, const char *<i>string</i>,</b> -<b> size_t <i>nmatch</i>, regmatch_t <i>pmatch</i>[], int <i>eflags</i>);</b> -<b> size_t regerror(int <i>errcode</i>, const regex_t *<i>preg</i>,</b> -<b> char *<i>errbuf</i>, size_t <i>errbuf_size</i>);</b> -<br> -<br> -<b>void regfree(regex_t *<i>preg</i>);</b> -</P> -<br><a name="SEC2" href="#TOC1">DESCRIPTION</a><br> -<P> -This set of functions provides a POSIX-style API for the PCRE regular -expression 8-bit library. See the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation for a description of PCRE's native API, which contains much -additional functionality. There is no POSIX-style wrapper for PCRE's 16-bit -and 32-bit library. -</P> -<P> -The functions described here are just wrapper functions that ultimately call -the PCRE native API. Their prototypes are defined in the <b>pcreposix.h</b> -header file, and on Unix systems the library itself is called -<b>pcreposix.a</b>, so can be accessed by adding <b>-lpcreposix</b> to the -command for linking an application that uses them. Because the POSIX functions -call the native ones, it is also necessary to add <b>-lpcre</b>. -</P> -<P> -I have implemented only those POSIX option bits that can be reasonably mapped -to PCRE native options. In addition, the option REG_EXTENDED is defined with -the value zero. This has no effect, but since programs that are written to the -POSIX interface often use it, this makes it easier to slot in PCRE as a -replacement library. Other POSIX options are not even defined. -</P> -<P> -There are also some other options that are not defined by POSIX. These have -been added at the request of users who want to make use of certain -PCRE-specific features via the POSIX calling interface. -</P> -<P> -When PCRE is called via these functions, it is only the API that is POSIX-like -in style. The syntax and semantics of the regular expressions themselves are -still those of Perl, subject to the setting of various PCRE options, as -described below. "POSIX-like in style" means that the API approximates to the -POSIX definition; it is not fully POSIX-compatible, and in multi-byte encoding -domains it is probably even less compatible. -</P> -<P> -The header for these functions is supplied as <b>pcreposix.h</b> to avoid any -potential clash with other POSIX libraries. It can, of course, be renamed or -aliased as <b>regex.h</b>, which is the "correct" name. It provides two -structure types, <i>regex_t</i> for compiled internal forms, and -<i>regmatch_t</i> for returning captured substrings. It also defines some -constants whose names start with "REG_"; these are used for setting options and -identifying error codes. -</P> -<br><a name="SEC3" href="#TOC1">COMPILING A PATTERN</a><br> -<P> -The function <b>regcomp()</b> is called to compile a pattern into an -internal form. The pattern is a C string terminated by a binary zero, and -is passed in the argument <i>pattern</i>. The <i>preg</i> argument is a pointer -to a <b>regex_t</b> structure that is used as a base for storing information -about the compiled regular expression. -</P> -<P> -The argument <i>cflags</i> is either zero, or contains one or more of the bits -defined by the following macros: -<pre> - REG_DOTALL -</pre> -The PCRE_DOTALL option is set when the regular expression is passed for -compilation to the native function. Note that REG_DOTALL is not part of the -POSIX standard. -<pre> - REG_ICASE -</pre> -The PCRE_CASELESS option is set when the regular expression is passed for -compilation to the native function. -<pre> - REG_NEWLINE -</pre> -The PCRE_MULTILINE option is set when the regular expression is passed for -compilation to the native function. Note that this does <i>not</i> mimic the -defined POSIX behaviour for REG_NEWLINE (see the following section). -<pre> - REG_NOSUB -</pre> -The PCRE_NO_AUTO_CAPTURE option is set when the regular expression is passed -for compilation to the native function. In addition, when a pattern that is -compiled with this flag is passed to <b>regexec()</b> for matching, the -<i>nmatch</i> and <i>pmatch</i> arguments are ignored, and no captured strings -are returned. -<pre> - REG_UCP -</pre> -The PCRE_UCP option is set when the regular expression is passed for -compilation to the native function. This causes PCRE to use Unicode properties -when matchine \d, \w, etc., instead of just recognizing ASCII values. Note -that REG_UTF8 is not part of the POSIX standard. -<pre> - REG_UNGREEDY -</pre> -The PCRE_UNGREEDY option is set when the regular expression is passed for -compilation to the native function. Note that REG_UNGREEDY is not part of the -POSIX standard. -<pre> - REG_UTF8 -</pre> -The PCRE_UTF8 option is set when the regular expression is passed for -compilation to the native function. This causes the pattern itself and all data -strings used for matching it to be treated as UTF-8 strings. Note that REG_UTF8 -is not part of the POSIX standard. -</P> -<P> -In the absence of these flags, no options are passed to the native function. -This means the the regex is compiled with PCRE default semantics. In -particular, the way it handles newline characters in the subject string is the -Perl way, not the POSIX way. Note that setting PCRE_MULTILINE has only -<i>some</i> of the effects specified for REG_NEWLINE. It does not affect the way -newlines are matched by . (they are not) or by a negative class such as [^a] -(they are). -</P> -<P> -The yield of <b>regcomp()</b> is zero on success, and non-zero otherwise. The -<i>preg</i> structure is filled in on success, and one member of the structure -is public: <i>re_nsub</i> contains the number of capturing subpatterns in -the regular expression. Various error codes are defined in the header file. -</P> -<P> -NOTE: If the yield of <b>regcomp()</b> is non-zero, you must not attempt to -use the contents of the <i>preg</i> structure. If, for example, you pass it to -<b>regexec()</b>, the result is undefined and your program is likely to crash. -</P> -<br><a name="SEC4" href="#TOC1">MATCHING NEWLINE CHARACTERS</a><br> -<P> -This area is not simple, because POSIX and Perl take different views of things. -It is not possible to get PCRE to obey POSIX semantics, but then PCRE was never -intended to be a POSIX engine. The following table lists the different -possibilities for matching newline characters in PCRE: -<pre> - Default Change with - - . matches newline no PCRE_DOTALL - newline matches [^a] yes not changeable - $ matches \n at end yes PCRE_DOLLARENDONLY - $ matches \n in middle no PCRE_MULTILINE - ^ matches \n in middle no PCRE_MULTILINE -</pre> -This is the equivalent table for POSIX: -<pre> - Default Change with - - . matches newline yes REG_NEWLINE - newline matches [^a] yes REG_NEWLINE - $ matches \n at end no REG_NEWLINE - $ matches \n in middle no REG_NEWLINE - ^ matches \n in middle no REG_NEWLINE -</pre> -PCRE's behaviour is the same as Perl's, except that there is no equivalent for -PCRE_DOLLAR_ENDONLY in Perl. In both PCRE and Perl, there is no way to stop -newline from matching [^a]. -</P> -<P> -The default POSIX newline handling can be obtained by setting PCRE_DOTALL and -PCRE_DOLLAR_ENDONLY, but there is no way to make PCRE behave exactly as for the -REG_NEWLINE action. -</P> -<br><a name="SEC5" href="#TOC1">MATCHING A PATTERN</a><br> -<P> -The function <b>regexec()</b> is called to match a compiled pattern <i>preg</i> -against a given <i>string</i>, which is by default terminated by a zero byte -(but see REG_STARTEND below), subject to the options in <i>eflags</i>. These can -be: -<pre> - REG_NOTBOL -</pre> -The PCRE_NOTBOL option is set when calling the underlying PCRE matching -function. -<pre> - REG_NOTEMPTY -</pre> -The PCRE_NOTEMPTY option is set when calling the underlying PCRE matching -function. Note that REG_NOTEMPTY is not part of the POSIX standard. However, -setting this option can give more POSIX-like behaviour in some situations. -<pre> - REG_NOTEOL -</pre> -The PCRE_NOTEOL option is set when calling the underlying PCRE matching -function. -<pre> - REG_STARTEND -</pre> -The string is considered to start at <i>string</i> + <i>pmatch[0].rm_so</i> and -to have a terminating NUL located at <i>string</i> + <i>pmatch[0].rm_eo</i> -(there need not actually be a NUL at that location), regardless of the value of -<i>nmatch</i>. This is a BSD extension, compatible with but not specified by -IEEE Standard 1003.2 (POSIX.2), and should be used with caution in software -intended to be portable to other systems. Note that a non-zero <i>rm_so</i> does -not imply REG_NOTBOL; REG_STARTEND affects only the location of the string, not -how it is matched. -</P> -<P> -If the pattern was compiled with the REG_NOSUB flag, no data about any matched -strings is returned. The <i>nmatch</i> and <i>pmatch</i> arguments of -<b>regexec()</b> are ignored. -</P> -<P> -If the value of <i>nmatch</i> is zero, or if the value <i>pmatch</i> is NULL, -no data about any matched strings is returned. -</P> -<P> -Otherwise,the portion of the string that was matched, and also any captured -substrings, are returned via the <i>pmatch</i> argument, which points to an -array of <i>nmatch</i> structures of type <i>regmatch_t</i>, containing the -members <i>rm_so</i> and <i>rm_eo</i>. These contain the offset to the first -character of each substring and the offset to the first character after the end -of each substring, respectively. The 0th element of the vector relates to the -entire portion of <i>string</i> that was matched; subsequent elements relate to -the capturing subpatterns of the regular expression. Unused entries in the -array have both structure members set to -1. -</P> -<P> -A successful match yields a zero return; various error codes are defined in the -header file, of which REG_NOMATCH is the "expected" failure code. -</P> -<br><a name="SEC6" href="#TOC1">ERROR MESSAGES</a><br> -<P> -The <b>regerror()</b> function maps a non-zero errorcode from either -<b>regcomp()</b> or <b>regexec()</b> to a printable message. If <i>preg</i> is not -NULL, the error should have arisen from the use of that structure. A message -terminated by a binary zero is placed in <i>errbuf</i>. The length of the -message, including the zero, is limited to <i>errbuf_size</i>. The yield of the -function is the size of buffer needed to hold the whole message. -</P> -<br><a name="SEC7" href="#TOC1">MEMORY USAGE</a><br> -<P> -Compiling a regular expression causes memory to be allocated and associated -with the <i>preg</i> structure. The function <b>regfree()</b> frees all such -memory, after which <i>preg</i> may no longer be used as a compiled expression. -</P> -<br><a name="SEC8" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC9" href="#TOC1">REVISION</a><br> -<P> -Last updated: 09 January 2012 -<br> -Copyright © 1997-2012 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcreprecompile.html b/plugins/Pcre16/docs/doc/html/pcreprecompile.html deleted file mode 100644 index decb1d6ce0..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcreprecompile.html +++ /dev/null @@ -1,163 +0,0 @@ -<html> -<head> -<title>pcreprecompile specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcreprecompile man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">SAVING AND RE-USING PRECOMPILED PCRE PATTERNS</a> -<li><a name="TOC2" href="#SEC2">SAVING A COMPILED PATTERN</a> -<li><a name="TOC3" href="#SEC3">RE-USING A PRECOMPILED PATTERN</a> -<li><a name="TOC4" href="#SEC4">COMPATIBILITY WITH DIFFERENT PCRE RELEASES</a> -<li><a name="TOC5" href="#SEC5">AUTHOR</a> -<li><a name="TOC6" href="#SEC6">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">SAVING AND RE-USING PRECOMPILED PCRE PATTERNS</a><br> -<P> -If you are running an application that uses a large number of regular -expression patterns, it may be useful to store them in a precompiled form -instead of having to compile them every time the application is run. -If you are not using any private character tables (see the -<a href="pcre_maketables.html"><b>pcre_maketables()</b></a> -documentation), this is relatively straightforward. If you are using private -tables, it is a little bit more complicated. However, if you are using the -just-in-time optimization feature, it is not possible to save and reload the -JIT data. -</P> -<P> -If you save compiled patterns to a file, you can copy them to a different host -and run them there. If the two hosts have different endianness (byte order), -you should run the <b>pcre[16|32]_pattern_to_host_byte_order()</b> function on the -new host before trying to match the pattern. The matching functions return -PCRE_ERROR_BADENDIANNESS if they detect a pattern with the wrong endianness. -</P> -<P> -Compiling regular expressions with one version of PCRE for use with a different -version is not guaranteed to work and may cause crashes, and saving and -restoring a compiled pattern loses any JIT optimization data. -</P> -<br><a name="SEC2" href="#TOC1">SAVING A COMPILED PATTERN</a><br> -<P> -The value returned by <b>pcre[16|32]_compile()</b> points to a single block of -memory that holds the compiled pattern and associated data. You can find the -length of this block in bytes by calling <b>pcre[16|32]_fullinfo()</b> with an -argument of PCRE_INFO_SIZE. You can then save the data in any appropriate -manner. Here is sample code for the 8-bit library that compiles a pattern and -writes it to a file. It assumes that the variable <i>fd</i> refers to a file -that is open for output: -<pre> - int erroroffset, rc, size; - char *error; - pcre *re; - - re = pcre_compile("my pattern", 0, &error, &erroroffset, NULL); - if (re == NULL) { ... handle errors ... } - rc = pcre_fullinfo(re, NULL, PCRE_INFO_SIZE, &size); - if (rc < 0) { ... handle errors ... } - rc = fwrite(re, 1, size, fd); - if (rc != size) { ... handle errors ... } -</pre> -In this example, the bytes that comprise the compiled pattern are copied -exactly. Note that this is binary data that may contain any of the 256 possible -byte values. On systems that make a distinction between binary and non-binary -data, be sure that the file is opened for binary output. -</P> -<P> -If you want to write more than one pattern to a file, you will have to devise a -way of separating them. For binary data, preceding each pattern with its length -is probably the most straightforward approach. Another possibility is to write -out the data in hexadecimal instead of binary, one pattern to a line. -</P> -<P> -Saving compiled patterns in a file is only one possible way of storing them for -later use. They could equally well be saved in a database, or in the memory of -some daemon process that passes them via sockets to the processes that want -them. -</P> -<P> -If the pattern has been studied, it is also possible to save the normal study -data in a similar way to the compiled pattern itself. However, if the -PCRE_STUDY_JIT_COMPILE was used, the just-in-time data that is created cannot -be saved because it is too dependent on the current environment. When studying -generates additional information, <b>pcre[16|32]_study()</b> returns a pointer to a -<b>pcre[16|32]_extra</b> data block. Its format is defined in the -<a href="pcreapi.html#extradata">section on matching a pattern</a> -in the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation. The <i>study_data</i> field points to the binary study data, and -this is what you must save (not the <b>pcre[16|32]_extra</b> block itself). The -length of the study data can be obtained by calling <b>pcre[16|32]_fullinfo()</b> -with an argument of PCRE_INFO_STUDYSIZE. Remember to check that -<b>pcre[16|32]_study()</b> did return a non-NULL value before trying to save the -study data. -</P> -<br><a name="SEC3" href="#TOC1">RE-USING A PRECOMPILED PATTERN</a><br> -<P> -Re-using a precompiled pattern is straightforward. Having reloaded it into main -memory, called <b>pcre[16|32]_pattern_to_host_byte_order()</b> if necessary, you -pass its pointer to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> in -the usual way. -</P> -<P> -However, if you passed a pointer to custom character tables when the pattern -was compiled (the <i>tableptr</i> argument of <b>pcre[16|32]_compile()</b>), you -must now pass a similar pointer to <b>pcre[16|32]_exec()</b> or -<b>pcre[16|32]_dfa_exec()</b>, because the value saved with the compiled pattern -will obviously be nonsense. A field in a <b>pcre[16|32]_extra()</b> block is used -to pass this data, as described in the -<a href="pcreapi.html#extradata">section on matching a pattern</a> -in the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation. -</P> -<P> -<b>Warning:</b> The tables that <b>pcre_exec()</b> and <b>pcre_dfa_exec()</b> use -must be the same as those that were used when the pattern was compiled. If this -is not the case, the behaviour is undefined. -</P> -<P> -If you did not provide custom character tables when the pattern was compiled, -the pointer in the compiled pattern is NULL, which causes the matching -functions to use PCRE's internal tables. Thus, you do not need to take any -special action at run time in this case. -</P> -<P> -If you saved study data with the compiled pattern, you need to create your own -<b>pcre[16|32]_extra</b> data block and set the <i>study_data</i> field to point -to the reloaded study data. You must also set the PCRE_EXTRA_STUDY_DATA bit in -the <i>flags</i> field to indicate that study data is present. Then pass the -<b>pcre[16|32]_extra</b> block to the matching function in the usual way. If the -pattern was studied for just-in-time optimization, that data cannot be saved, -and so is lost by a save/restore cycle. -</P> -<br><a name="SEC4" href="#TOC1">COMPATIBILITY WITH DIFFERENT PCRE RELEASES</a><br> -<P> -In general, it is safest to recompile all saved patterns when you update to a -new PCRE release, though not all updates actually require this. -</P> -<br><a name="SEC5" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC6" href="#TOC1">REVISION</a><br> -<P> -Last updated: 12 November 2013 -<br> -Copyright © 1997-2013 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcresample.html b/plugins/Pcre16/docs/doc/html/pcresample.html deleted file mode 100644 index aca9184e00..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcresample.html +++ /dev/null @@ -1,110 +0,0 @@ -<html> -<head> -<title>pcresample specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcresample man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -PCRE SAMPLE PROGRAM -</b><br> -<P> -A simple, complete demonstration program, to get you started with using PCRE, -is supplied in the file <i>pcredemo.c</i> in the PCRE distribution. A listing of -this program is given in the -<a href="pcredemo.html"><b>pcredemo</b></a> -documentation. If you do not have a copy of the PCRE distribution, you can save -this listing to re-create <i>pcredemo.c</i>. -</P> -<P> -The demonstration program, which uses the original PCRE 8-bit library, compiles -the regular expression that is its first argument, and matches it against the -subject string in its second argument. No PCRE options are set, and default -character tables are used. If matching succeeds, the program outputs the -portion of the subject that matched, together with the contents of any captured -substrings. -</P> -<P> -If the -g option is given on the command line, the program then goes on to -check for further matches of the same regular expression in the same subject -string. The logic is a little bit tricky because of the possibility of matching -an empty string. Comments in the code explain what is going on. -</P> -<P> -If PCRE is installed in the standard include and library directories for your -operating system, you should be able to compile the demonstration program using -this command: -<pre> - gcc -o pcredemo pcredemo.c -lpcre -</pre> -If PCRE is installed elsewhere, you may need to add additional options to the -command line. For example, on a Unix-like system that has PCRE installed in -<i>/usr/local</i>, you can compile the demonstration program using a command -like this: -<pre> - gcc -o pcredemo -I/usr/local/include pcredemo.c -L/usr/local/lib -lpcre -</pre> -In a Windows environment, if you want to statically link the program against a -non-dll <b>pcre.a</b> file, you must uncomment the line that defines PCRE_STATIC -before including <b>pcre.h</b>, because otherwise the <b>pcre_malloc()</b> and -<b>pcre_free()</b> exported functions will be declared -<b>__declspec(dllimport)</b>, with unwanted results. -</P> -<P> -Once you have compiled and linked the demonstration program, you can run simple -tests like this: -<pre> - ./pcredemo 'cat|dog' 'the cat sat on the mat' - ./pcredemo -g 'cat|dog' 'the dog sat on the cat' -</pre> -Note that there is a much more comprehensive test program, called -<a href="pcretest.html"><b>pcretest</b>,</a> -which supports many more facilities for testing regular expressions and both -PCRE libraries. The -<a href="pcredemo.html"><b>pcredemo</b></a> -program is provided as a simple coding example. -</P> -<P> -If you try to run -<a href="pcredemo.html"><b>pcredemo</b></a> -when PCRE is not installed in the standard library directory, you may get an -error like this on some operating systems (e.g. Solaris): -<pre> - ld.so.1: a.out: fatal: libpcre.so.0: open failed: No such file or directory -</pre> -This is caused by the way shared library support works on those systems. You -need to add -<pre> - -R/usr/local/lib -</pre> -(for example) to the compile command to get round this problem. -</P> -<br><b> -AUTHOR -</b><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><b> -REVISION -</b><br> -<P> -Last updated: 10 January 2012 -<br> -Copyright © 1997-2012 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcrestack.html b/plugins/Pcre16/docs/doc/html/pcrestack.html deleted file mode 100644 index af6406d070..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcrestack.html +++ /dev/null @@ -1,225 +0,0 @@ -<html> -<head> -<title>pcrestack specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcrestack man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -PCRE DISCUSSION OF STACK USAGE -</b><br> -<P> -When you call <b>pcre[16|32]_exec()</b>, it makes use of an internal function -called <b>match()</b>. This calls itself recursively at branch points in the -pattern, in order to remember the state of the match so that it can back up and -try a different alternative if the first one fails. As matching proceeds deeper -and deeper into the tree of possibilities, the recursion depth increases. The -<b>match()</b> function is also called in other circumstances, for example, -whenever a parenthesized sub-pattern is entered, and in certain cases of -repetition. -</P> -<P> -Not all calls of <b>match()</b> increase the recursion depth; for an item such -as a* it may be called several times at the same level, after matching -different numbers of a's. Furthermore, in a number of cases where the result of -the recursive call would immediately be passed back as the result of the -current call (a "tail recursion"), the function is just restarted instead. -</P> -<P> -The above comments apply when <b>pcre[16|32]_exec()</b> is run in its normal -interpretive manner. If the pattern was studied with the -PCRE_STUDY_JIT_COMPILE option, and just-in-time compiling was successful, and -the options passed to <b>pcre[16|32]_exec()</b> were not incompatible, the matching -process uses the JIT-compiled code instead of the <b>match()</b> function. In -this case, the memory requirements are handled entirely differently. See the -<a href="pcrejit.html"><b>pcrejit</b></a> -documentation for details. -</P> -<P> -The <b>pcre[16|32]_dfa_exec()</b> function operates in an entirely different way, -and uses recursion only when there is a regular expression recursion or -subroutine call in the pattern. This includes the processing of assertion and -"once-only" subpatterns, which are handled like subroutine calls. Normally, -these are never very deep, and the limit on the complexity of -<b>pcre[16|32]_dfa_exec()</b> is controlled by the amount of workspace it is given. -However, it is possible to write patterns with runaway infinite recursions; -such patterns will cause <b>pcre[16|32]_dfa_exec()</b> to run out of stack. At -present, there is no protection against this. -</P> -<P> -The comments that follow do NOT apply to <b>pcre[16|32]_dfa_exec()</b>; they are -relevant only for <b>pcre[16|32]_exec()</b> without the JIT optimization. -</P> -<br><b> -Reducing <b>pcre[16|32]_exec()</b>'s stack usage -</b><br> -<P> -Each time that <b>match()</b> is actually called recursively, it uses memory -from the process stack. For certain kinds of pattern and data, very large -amounts of stack may be needed, despite the recognition of "tail recursion". -You can often reduce the amount of recursion, and therefore the amount of stack -used, by modifying the pattern that is being matched. Consider, for example, -this pattern: -<pre> - ([^<]|<(?!inet))+ -</pre> -It matches from wherever it starts until it encounters "<inet" or the end of -the data, and is the kind of pattern that might be used when processing an XML -file. Each iteration of the outer parentheses matches either one character that -is not "<" or a "<" that is not followed by "inet". However, each time a -parenthesis is processed, a recursion occurs, so this formulation uses a stack -frame for each matched character. For a long string, a lot of stack is -required. Consider now this rewritten pattern, which matches exactly the same -strings: -<pre> - ([^<]++|<(?!inet))+ -</pre> -This uses very much less stack, because runs of characters that do not contain -"<" are "swallowed" in one item inside the parentheses. Recursion happens only -when a "<" character that is not followed by "inet" is encountered (and we -assume this is relatively rare). A possessive quantifier is used to stop any -backtracking into the runs of non-"<" characters, but that is not related to -stack usage. -</P> -<P> -This example shows that one way of avoiding stack problems when matching long -subject strings is to write repeated parenthesized subpatterns to match more -than one character whenever possible. -</P> -<br><b> -Compiling PCRE to use heap instead of stack for <b>pcre[16|32]_exec()</b> -</b><br> -<P> -In environments where stack memory is constrained, you might want to compile -PCRE to use heap memory instead of stack for remembering back-up points when -<b>pcre[16|32]_exec()</b> is running. This makes it run a lot more slowly, however. -Details of how to do this are given in the -<a href="pcrebuild.html"><b>pcrebuild</b></a> -documentation. When built in this way, instead of using the stack, PCRE obtains -and frees memory by calling the functions that are pointed to by the -<b>pcre[16|32]_stack_malloc</b> and <b>pcre[16|32]_stack_free</b> variables. By -default, these point to <b>malloc()</b> and <b>free()</b>, but you can replace -the pointers to cause PCRE to use your own functions. Since the block sizes are -always the same, and are always freed in reverse order, it may be possible to -implement customized memory handlers that are more efficient than the standard -functions. -</P> -<br><b> -Limiting <b>pcre[16|32]_exec()</b>'s stack usage -</b><br> -<P> -You can set limits on the number of times that <b>match()</b> is called, both in -total and recursively. If a limit is exceeded, <b>pcre[16|32]_exec()</b> returns an -error code. Setting suitable limits should prevent it from running out of -stack. The default values of the limits are very large, and unlikely ever to -operate. They can be changed when PCRE is built, and they can also be set when -<b>pcre[16|32]_exec()</b> is called. For details of these interfaces, see the -<a href="pcrebuild.html"><b>pcrebuild</b></a> -documentation and the -<a href="pcreapi.html#extradata">section on extra data for <b>pcre[16|32]_exec()</b></a> -in the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation. -</P> -<P> -As a very rough rule of thumb, you should reckon on about 500 bytes per -recursion. Thus, if you want to limit your stack usage to 8Mb, you should set -the limit at 16000 recursions. A 64Mb stack, on the other hand, can support -around 128000 recursions. -</P> -<P> -In Unix-like environments, the <b>pcretest</b> test program has a command line -option (<b>-S</b>) that can be used to increase the size of its stack. As long -as the stack is large enough, another option (<b>-M</b>) can be used to find the -smallest limits that allow a particular pattern to match a given subject -string. This is done by calling <b>pcre[16|32]_exec()</b> repeatedly with different -limits. -</P> -<br><b> -Obtaining an estimate of stack usage -</b><br> -<P> -The actual amount of stack used per recursion can vary quite a lot, depending -on the compiler that was used to build PCRE and the optimization or debugging -options that were set for it. The rule of thumb value of 500 bytes mentioned -above may be larger or smaller than what is actually needed. A better -approximation can be obtained by running this command: -<pre> - pcretest -m -C -</pre> -The <b>-C</b> option causes <b>pcretest</b> to output information about the -options with which PCRE was compiled. When <b>-m</b> is also given (before -<b>-C</b>), information about stack use is given in a line like this: -<pre> - Match recursion uses stack: approximate frame size = 640 bytes -</pre> -The value is approximate because some recursions need a bit more (up to perhaps -16 more bytes). -</P> -<P> -If the above command is given when PCRE is compiled to use the heap instead of -the stack for recursion, the value that is output is the size of each block -that is obtained from the heap. -</P> -<br><b> -Changing stack size in Unix-like systems -</b><br> -<P> -In Unix-like environments, there is not often a problem with the stack unless -very long strings are involved, though the default limit on stack size varies -from system to system. Values from 8Mb to 64Mb are common. You can find your -default limit by running the command: -<pre> - ulimit -s -</pre> -Unfortunately, the effect of running out of stack is often SIGSEGV, though -sometimes a more explicit error message is given. You can normally increase the -limit on stack size by code such as this: -<pre> - struct rlimit rlim; - getrlimit(RLIMIT_STACK, &rlim); - rlim.rlim_cur = 100*1024*1024; - setrlimit(RLIMIT_STACK, &rlim); -</pre> -This reads the current limits (soft and hard) using <b>getrlimit()</b>, then -attempts to increase the soft limit to 100Mb using <b>setrlimit()</b>. You must -do this before calling <b>pcre[16|32]_exec()</b>. -</P> -<br><b> -Changing stack size in Mac OS X -</b><br> -<P> -Using <b>setrlimit()</b>, as described above, should also work on Mac OS X. It -is also possible to set a stack size when linking a program. There is a -discussion about stack sizes in Mac OS X at this web site: -<a href="http://developer.apple.com/qa/qa2005/qa1419.html">http://developer.apple.com/qa/qa2005/qa1419.html.</a> -</P> -<br><b> -AUTHOR -</b><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><b> -REVISION -</b><br> -<P> -Last updated: 24 June 2012 -<br> -Copyright © 1997-2012 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcresyntax.html b/plugins/Pcre16/docs/doc/html/pcresyntax.html deleted file mode 100644 index 89f35737b4..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcresyntax.html +++ /dev/null @@ -1,538 +0,0 @@ -<html> -<head> -<title>pcresyntax specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcresyntax man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">PCRE REGULAR EXPRESSION SYNTAX SUMMARY</a> -<li><a name="TOC2" href="#SEC2">QUOTING</a> -<li><a name="TOC3" href="#SEC3">CHARACTERS</a> -<li><a name="TOC4" href="#SEC4">CHARACTER TYPES</a> -<li><a name="TOC5" href="#SEC5">GENERAL CATEGORY PROPERTIES FOR \p and \P</a> -<li><a name="TOC6" href="#SEC6">PCRE SPECIAL CATEGORY PROPERTIES FOR \p and \P</a> -<li><a name="TOC7" href="#SEC7">SCRIPT NAMES FOR \p AND \P</a> -<li><a name="TOC8" href="#SEC8">CHARACTER CLASSES</a> -<li><a name="TOC9" href="#SEC9">QUANTIFIERS</a> -<li><a name="TOC10" href="#SEC10">ANCHORS AND SIMPLE ASSERTIONS</a> -<li><a name="TOC11" href="#SEC11">MATCH POINT RESET</a> -<li><a name="TOC12" href="#SEC12">ALTERNATION</a> -<li><a name="TOC13" href="#SEC13">CAPTURING</a> -<li><a name="TOC14" href="#SEC14">ATOMIC GROUPS</a> -<li><a name="TOC15" href="#SEC15">COMMENT</a> -<li><a name="TOC16" href="#SEC16">OPTION SETTING</a> -<li><a name="TOC17" href="#SEC17">NEWLINE CONVENTION</a> -<li><a name="TOC18" href="#SEC18">WHAT \R MATCHES</a> -<li><a name="TOC19" href="#SEC19">LOOKAHEAD AND LOOKBEHIND ASSERTIONS</a> -<li><a name="TOC20" href="#SEC20">BACKREFERENCES</a> -<li><a name="TOC21" href="#SEC21">SUBROUTINE REFERENCES (POSSIBLY RECURSIVE)</a> -<li><a name="TOC22" href="#SEC22">CONDITIONAL PATTERNS</a> -<li><a name="TOC23" href="#SEC23">BACKTRACKING CONTROL</a> -<li><a name="TOC24" href="#SEC24">CALLOUTS</a> -<li><a name="TOC25" href="#SEC25">SEE ALSO</a> -<li><a name="TOC26" href="#SEC26">AUTHOR</a> -<li><a name="TOC27" href="#SEC27">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">PCRE REGULAR EXPRESSION SYNTAX SUMMARY</a><br> -<P> -The full syntax and semantics of the regular expressions that are supported by -PCRE are described in the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -documentation. This document contains a quick-reference summary of the syntax. -</P> -<br><a name="SEC2" href="#TOC1">QUOTING</a><br> -<P> -<pre> - \x where x is non-alphanumeric is a literal x - \Q...\E treat enclosed characters as literal -</PRE> -</P> -<br><a name="SEC3" href="#TOC1">CHARACTERS</a><br> -<P> -<pre> - \a alarm, that is, the BEL character (hex 07) - \cx "control-x", where x is any ASCII character - \e escape (hex 1B) - \f form feed (hex 0C) - \n newline (hex 0A) - \r carriage return (hex 0D) - \t tab (hex 09) - \0dd character with octal code 0dd - \ddd character with octal code ddd, or backreference - \o{ddd..} character with octal code ddd.. - \xhh character with hex code hh - \x{hhh..} character with hex code hhh.. -</pre> -Note that \0dd is always an octal code, and that \8 and \9 are the literal -characters "8" and "9". -</P> -<br><a name="SEC4" href="#TOC1">CHARACTER TYPES</a><br> -<P> -<pre> - . any character except newline; - in dotall mode, any character whatsoever - \C one data unit, even in UTF mode (best avoided) - \d a decimal digit - \D a character that is not a decimal digit - \h a horizontal white space character - \H a character that is not a horizontal white space character - \N a character that is not a newline - \p{<i>xx</i>} a character with the <i>xx</i> property - \P{<i>xx</i>} a character without the <i>xx</i> property - \R a newline sequence - \s a white space character - \S a character that is not a white space character - \v a vertical white space character - \V a character that is not a vertical white space character - \w a "word" character - \W a "non-word" character - \X a Unicode extended grapheme cluster -</pre> -By default, \d, \s, and \w match only ASCII characters, even in UTF-8 mode -or in the 16- bit and 32-bit libraries. However, if locale-specific matching is -happening, \s and \w may also match characters with code points in the range -128-255. If the PCRE_UCP option is set, the behaviour of these escape sequences -is changed to use Unicode properties and they match many more characters. -</P> -<br><a name="SEC5" href="#TOC1">GENERAL CATEGORY PROPERTIES FOR \p and \P</a><br> -<P> -<pre> - C Other - Cc Control - Cf Format - Cn Unassigned - Co Private use - Cs Surrogate - - L Letter - Ll Lower case letter - Lm Modifier letter - Lo Other letter - Lt Title case letter - Lu Upper case letter - L& Ll, Lu, or Lt - - M Mark - Mc Spacing mark - Me Enclosing mark - Mn Non-spacing mark - - N Number - Nd Decimal number - Nl Letter number - No Other number - - P Punctuation - Pc Connector punctuation - Pd Dash punctuation - Pe Close punctuation - Pf Final punctuation - Pi Initial punctuation - Po Other punctuation - Ps Open punctuation - - S Symbol - Sc Currency symbol - Sk Modifier symbol - Sm Mathematical symbol - So Other symbol - - Z Separator - Zl Line separator - Zp Paragraph separator - Zs Space separator -</PRE> -</P> -<br><a name="SEC6" href="#TOC1">PCRE SPECIAL CATEGORY PROPERTIES FOR \p and \P</a><br> -<P> -<pre> - Xan Alphanumeric: union of properties L and N - Xps POSIX space: property Z or tab, NL, VT, FF, CR - Xsp Perl space: property Z or tab, NL, VT, FF, CR - Xuc Univerally-named character: one that can be - represented by a Universal Character Name - Xwd Perl word: property Xan or underscore -</pre> -Perl and POSIX space are now the same. Perl added VT to its space character set -at release 5.18 and PCRE changed at release 8.34. -</P> -<br><a name="SEC7" href="#TOC1">SCRIPT NAMES FOR \p AND \P</a><br> -<P> -Arabic, -Armenian, -Avestan, -Balinese, -Bamum, -Batak, -Bengali, -Bopomofo, -Brahmi, -Braille, -Buginese, -Buhid, -Canadian_Aboriginal, -Carian, -Chakma, -Cham, -Cherokee, -Common, -Coptic, -Cuneiform, -Cypriot, -Cyrillic, -Deseret, -Devanagari, -Egyptian_Hieroglyphs, -Ethiopic, -Georgian, -Glagolitic, -Gothic, -Greek, -Gujarati, -Gurmukhi, -Han, -Hangul, -Hanunoo, -Hebrew, -Hiragana, -Imperial_Aramaic, -Inherited, -Inscriptional_Pahlavi, -Inscriptional_Parthian, -Javanese, -Kaithi, -Kannada, -Katakana, -Kayah_Li, -Kharoshthi, -Khmer, -Lao, -Latin, -Lepcha, -Limbu, -Linear_B, -Lisu, -Lycian, -Lydian, -Malayalam, -Mandaic, -Meetei_Mayek, -Meroitic_Cursive, -Meroitic_Hieroglyphs, -Miao, -Mongolian, -Myanmar, -New_Tai_Lue, -Nko, -Ogham, -Old_Italic, -Old_Persian, -Old_South_Arabian, -Old_Turkic, -Ol_Chiki, -Oriya, -Osmanya, -Phags_Pa, -Phoenician, -Rejang, -Runic, -Samaritan, -Saurashtra, -Sharada, -Shavian, -Sinhala, -Sora_Sompeng, -Sundanese, -Syloti_Nagri, -Syriac, -Tagalog, -Tagbanwa, -Tai_Le, -Tai_Tham, -Tai_Viet, -Takri, -Tamil, -Telugu, -Thaana, -Thai, -Tibetan, -Tifinagh, -Ugaritic, -Vai, -Yi. -</P> -<br><a name="SEC8" href="#TOC1">CHARACTER CLASSES</a><br> -<P> -<pre> - [...] positive character class - [^...] negative character class - [x-y] range (can be used for hex characters) - [[:xxx:]] positive POSIX named set - [[:^xxx:]] negative POSIX named set - - alnum alphanumeric - alpha alphabetic - ascii 0-127 - blank space or tab - cntrl control character - digit decimal digit - graph printing, excluding space - lower lower case letter - print printing, including space - punct printing, excluding alphanumeric - space white space - upper upper case letter - word same as \w - xdigit hexadecimal digit -</pre> -In PCRE, POSIX character set names recognize only ASCII characters by default, -but some of them use Unicode properties if PCRE_UCP is set. You can use -\Q...\E inside a character class. -</P> -<br><a name="SEC9" href="#TOC1">QUANTIFIERS</a><br> -<P> -<pre> - ? 0 or 1, greedy - ?+ 0 or 1, possessive - ?? 0 or 1, lazy - * 0 or more, greedy - *+ 0 or more, possessive - *? 0 or more, lazy - + 1 or more, greedy - ++ 1 or more, possessive - +? 1 or more, lazy - {n} exactly n - {n,m} at least n, no more than m, greedy - {n,m}+ at least n, no more than m, possessive - {n,m}? at least n, no more than m, lazy - {n,} n or more, greedy - {n,}+ n or more, possessive - {n,}? n or more, lazy -</PRE> -</P> -<br><a name="SEC10" href="#TOC1">ANCHORS AND SIMPLE ASSERTIONS</a><br> -<P> -<pre> - \b word boundary - \B not a word boundary - ^ start of subject - also after internal newline in multiline mode - \A start of subject - $ end of subject - also before newline at end of subject - also before internal newline in multiline mode - \Z end of subject - also before newline at end of subject - \z end of subject - \G first matching position in subject -</PRE> -</P> -<br><a name="SEC11" href="#TOC1">MATCH POINT RESET</a><br> -<P> -<pre> - \K reset start of match -</pre> -\K is honoured in positive assertions, but ignored in negative ones. -</P> -<br><a name="SEC12" href="#TOC1">ALTERNATION</a><br> -<P> -<pre> - expr|expr|expr... -</PRE> -</P> -<br><a name="SEC13" href="#TOC1">CAPTURING</a><br> -<P> -<pre> - (...) capturing group - (?<name>...) named capturing group (Perl) - (?'name'...) named capturing group (Perl) - (?P<name>...) named capturing group (Python) - (?:...) non-capturing group - (?|...) non-capturing group; reset group numbers for - capturing groups in each alternative -</PRE> -</P> -<br><a name="SEC14" href="#TOC1">ATOMIC GROUPS</a><br> -<P> -<pre> - (?>...) atomic, non-capturing group -</PRE> -</P> -<br><a name="SEC15" href="#TOC1">COMMENT</a><br> -<P> -<pre> - (?#....) comment (not nestable) -</PRE> -</P> -<br><a name="SEC16" href="#TOC1">OPTION SETTING</a><br> -<P> -<pre> - (?i) caseless - (?J) allow duplicate names - (?m) multiline - (?s) single line (dotall) - (?U) default ungreedy (lazy) - (?x) extended (ignore white space) - (?-...) unset option(s) -</pre> -The following are recognized only at the very start of a pattern or after one -of the newline or \R options with similar syntax. More than one of them may -appear. -<pre> - (*LIMIT_MATCH=d) set the match limit to d (decimal number) - (*LIMIT_RECURSION=d) set the recursion limit to d (decimal number) - (*NO_AUTO_POSSESS) no auto-possessification (PCRE_NO_AUTO_POSSESS) - (*NO_START_OPT) no start-match optimization (PCRE_NO_START_OPTIMIZE) - (*UTF8) set UTF-8 mode: 8-bit library (PCRE_UTF8) - (*UTF16) set UTF-16 mode: 16-bit library (PCRE_UTF16) - (*UTF32) set UTF-32 mode: 32-bit library (PCRE_UTF32) - (*UTF) set appropriate UTF mode for the library in use - (*UCP) set PCRE_UCP (use Unicode properties for \d etc) -</pre> -Note that LIMIT_MATCH and LIMIT_RECURSION can only reduce the value of the -limits set by the caller of pcre_exec(), not increase them. -</P> -<br><a name="SEC17" href="#TOC1">NEWLINE CONVENTION</a><br> -<P> -These are recognized only at the very start of the pattern or after option -settings with a similar syntax. -<pre> - (*CR) carriage return only - (*LF) linefeed only - (*CRLF) carriage return followed by linefeed - (*ANYCRLF) all three of the above - (*ANY) any Unicode newline sequence -</PRE> -</P> -<br><a name="SEC18" href="#TOC1">WHAT \R MATCHES</a><br> -<P> -These are recognized only at the very start of the pattern or after option -setting with a similar syntax. -<pre> - (*BSR_ANYCRLF) CR, LF, or CRLF - (*BSR_UNICODE) any Unicode newline sequence -</PRE> -</P> -<br><a name="SEC19" href="#TOC1">LOOKAHEAD AND LOOKBEHIND ASSERTIONS</a><br> -<P> -<pre> - (?=...) positive look ahead - (?!...) negative look ahead - (?<=...) positive look behind - (?<!...) negative look behind -</pre> -Each top-level branch of a look behind must be of a fixed length. -</P> -<br><a name="SEC20" href="#TOC1">BACKREFERENCES</a><br> -<P> -<pre> - \n reference by number (can be ambiguous) - \gn reference by number - \g{n} reference by number - \g{-n} relative reference by number - \k<name> reference by name (Perl) - \k'name' reference by name (Perl) - \g{name} reference by name (Perl) - \k{name} reference by name (.NET) - (?P=name) reference by name (Python) -</PRE> -</P> -<br><a name="SEC21" href="#TOC1">SUBROUTINE REFERENCES (POSSIBLY RECURSIVE)</a><br> -<P> -<pre> - (?R) recurse whole pattern - (?n) call subpattern by absolute number - (?+n) call subpattern by relative number - (?-n) call subpattern by relative number - (?&name) call subpattern by name (Perl) - (?P>name) call subpattern by name (Python) - \g<name> call subpattern by name (Oniguruma) - \g'name' call subpattern by name (Oniguruma) - \g<n> call subpattern by absolute number (Oniguruma) - \g'n' call subpattern by absolute number (Oniguruma) - \g<+n> call subpattern by relative number (PCRE extension) - \g'+n' call subpattern by relative number (PCRE extension) - \g<-n> call subpattern by relative number (PCRE extension) - \g'-n' call subpattern by relative number (PCRE extension) -</PRE> -</P> -<br><a name="SEC22" href="#TOC1">CONDITIONAL PATTERNS</a><br> -<P> -<pre> - (?(condition)yes-pattern) - (?(condition)yes-pattern|no-pattern) - - (?(n)... absolute reference condition - (?(+n)... relative reference condition - (?(-n)... relative reference condition - (?(<name>)... named reference condition (Perl) - (?('name')... named reference condition (Perl) - (?(name)... named reference condition (PCRE) - (?(R)... overall recursion condition - (?(Rn)... specific group recursion condition - (?(R&name)... specific recursion condition - (?(DEFINE)... define subpattern for reference - (?(assert)... assertion condition -</PRE> -</P> -<br><a name="SEC23" href="#TOC1">BACKTRACKING CONTROL</a><br> -<P> -The following act immediately they are reached: -<pre> - (*ACCEPT) force successful match - (*FAIL) force backtrack; synonym (*F) - (*MARK:NAME) set name to be passed back; synonym (*:NAME) -</pre> -The following act only when a subsequent match failure causes a backtrack to -reach them. They all force a match failure, but they differ in what happens -afterwards. Those that advance the start-of-match point do so only if the -pattern is not anchored. -<pre> - (*COMMIT) overall failure, no advance of starting point - (*PRUNE) advance to next starting character - (*PRUNE:NAME) equivalent to (*MARK:NAME)(*PRUNE) - (*SKIP) advance to current matching position - (*SKIP:NAME) advance to position corresponding to an earlier - (*MARK:NAME); if not found, the (*SKIP) is ignored - (*THEN) local failure, backtrack to next alternation - (*THEN:NAME) equivalent to (*MARK:NAME)(*THEN) -</PRE> -</P> -<br><a name="SEC24" href="#TOC1">CALLOUTS</a><br> -<P> -<pre> - (?C) callout - (?Cn) callout with data n -</PRE> -</P> -<br><a name="SEC25" href="#TOC1">SEE ALSO</a><br> -<P> -<b>pcrepattern</b>(3), <b>pcreapi</b>(3), <b>pcrecallout</b>(3), -<b>pcrematching</b>(3), <b>pcre</b>(3). -</P> -<br><a name="SEC26" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC27" href="#TOC1">REVISION</a><br> -<P> -Last updated: 08 January 2014 -<br> -Copyright © 1997-2014 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcretest.html b/plugins/Pcre16/docs/doc/html/pcretest.html deleted file mode 100644 index 839fabf189..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcretest.html +++ /dev/null @@ -1,1158 +0,0 @@ -<html> -<head> -<title>pcretest specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcretest man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<ul> -<li><a name="TOC1" href="#SEC1">SYNOPSIS</a> -<li><a name="TOC2" href="#SEC2">INPUT DATA FORMAT</a> -<li><a name="TOC3" href="#SEC3">PCRE's 8-BIT, 16-BIT AND 32-BIT LIBRARIES</a> -<li><a name="TOC4" href="#SEC4">COMMAND LINE OPTIONS</a> -<li><a name="TOC5" href="#SEC5">DESCRIPTION</a> -<li><a name="TOC6" href="#SEC6">PATTERN MODIFIERS</a> -<li><a name="TOC7" href="#SEC7">DATA LINES</a> -<li><a name="TOC8" href="#SEC8">THE ALTERNATIVE MATCHING FUNCTION</a> -<li><a name="TOC9" href="#SEC9">DEFAULT OUTPUT FROM PCRETEST</a> -<li><a name="TOC10" href="#SEC10">OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION</a> -<li><a name="TOC11" href="#SEC11">RESTARTING AFTER A PARTIAL MATCH</a> -<li><a name="TOC12" href="#SEC12">CALLOUTS</a> -<li><a name="TOC13" href="#SEC13">NON-PRINTING CHARACTERS</a> -<li><a name="TOC14" href="#SEC14">SAVING AND RELOADING COMPILED PATTERNS</a> -<li><a name="TOC15" href="#SEC15">SEE ALSO</a> -<li><a name="TOC16" href="#SEC16">AUTHOR</a> -<li><a name="TOC17" href="#SEC17">REVISION</a> -</ul> -<br><a name="SEC1" href="#TOC1">SYNOPSIS</a><br> -<P> -<b>pcretest [options] [input file [output file]]</b> -<br> -<br> -<b>pcretest</b> was written as a test program for the PCRE regular expression -library itself, but it can also be used for experimenting with regular -expressions. This document describes the features of the test program; for -details of the regular expressions themselves, see the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -documentation. For details of the PCRE library function calls and their -options, see the -<a href="pcreapi.html"><b>pcreapi</b></a> -, -<a href="pcre16.html"><b>pcre16</b></a> -and -<a href="pcre32.html"><b>pcre32</b></a> -documentation. -</P> -<P> -The input for <b>pcretest</b> is a sequence of regular expression patterns and -strings to be matched, as described below. The output shows the result of each -match. Options on the command line and the patterns control PCRE options and -exactly what is output. -</P> -<P> -As PCRE has evolved, it has acquired many different features, and as a result, -<b>pcretest</b> now has rather a lot of obscure options for testing every -possible feature. Some of these options are specifically designed for use in -conjunction with the test script and data files that are distributed as part of -PCRE, and are unlikely to be of use otherwise. They are all documented here, -but without much justification. -</P> -<br><a name="SEC2" href="#TOC1">INPUT DATA FORMAT</a><br> -<P> -Input to <b>pcretest</b> is processed line by line, either by calling the C -library's <b>fgets()</b> function, or via the <b>libreadline</b> library (see -below). In Unix-like environments, <b>fgets()</b> treats any bytes other than -newline as data characters. However, in some Windows environments character 26 -(hex 1A) causes an immediate end of file, and no further data is read. For -maximum portability, therefore, it is safest to use only ASCII characters in -<b>pcretest</b> input files. -</P> -<br><a name="SEC3" href="#TOC1">PCRE's 8-BIT, 16-BIT AND 32-BIT LIBRARIES</a><br> -<P> -From release 8.30, two separate PCRE libraries can be built. The original one -supports 8-bit character strings, whereas the newer 16-bit library supports -character strings encoded in 16-bit units. From release 8.32, a third library -can be built, supporting character strings encoded in 32-bit units. The -<b>pcretest</b> program can be used to test all three libraries. However, it is -itself still an 8-bit program, reading 8-bit input and writing 8-bit output. -When testing the 16-bit or 32-bit library, the patterns and data strings are -converted to 16- or 32-bit format before being passed to the PCRE library -functions. Results are converted to 8-bit for output. -</P> -<P> -References to functions and structures of the form <b>pcre[16|32]_xx</b> below -mean "<b>pcre_xx</b> when using the 8-bit library, <b>pcre16_xx</b> when using -the 16-bit library, or <b>pcre32_xx</b> when using the 32-bit library". -</P> -<br><a name="SEC4" href="#TOC1">COMMAND LINE OPTIONS</a><br> -<P> -<b>-8</b> -If both the 8-bit library has been built, this option causes the 8-bit library -to be used (which is the default); if the 8-bit library has not been built, -this option causes an error. -</P> -<P> -<b>-16</b> -If both the 8-bit or the 32-bit, and the 16-bit libraries have been built, this -option causes the 16-bit library to be used. If only the 16-bit library has been -built, this is the default (so has no effect). If only the 8-bit or the 32-bit -library has been built, this option causes an error. -</P> -<P> -<b>-32</b> -If both the 8-bit or the 16-bit, and the 32-bit libraries have been built, this -option causes the 32-bit library to be used. If only the 32-bit library has been -built, this is the default (so has no effect). If only the 8-bit or the 16-bit -library has been built, this option causes an error. -</P> -<P> -<b>-b</b> -Behave as if each pattern has the <b>/B</b> (show byte code) modifier; the -internal form is output after compilation. -</P> -<P> -<b>-C</b> -Output the version number of the PCRE library, and all available information -about the optional features that are included, and then exit with zero exit -code. All other options are ignored. -</P> -<P> -<b>-C</b> <i>option</i> -Output information about a specific build-time option, then exit. This -functionality is intended for use in scripts such as <b>RunTest</b>. The -following options output the value and set the exit code as indicated: -<pre> - ebcdic-nl the code for LF (= NL) in an EBCDIC environment: - 0x15 or 0x25 - 0 if used in an ASCII environment - exit code is always 0 - linksize the configured internal link size (2, 3, or 4) - exit code is set to the link size - newline the default newline setting: - CR, LF, CRLF, ANYCRLF, or ANY - exit code is always 0 - bsr the default setting for what \R matches: - ANYCRLF or ANY - exit code is always 0 -</pre> -The following options output 1 for true or 0 for false, and set the exit code -to the same value: -<pre> - ebcdic compiled for an EBCDIC environment - jit just-in-time support is available - pcre16 the 16-bit library was built - pcre32 the 32-bit library was built - pcre8 the 8-bit library was built - ucp Unicode property support is available - utf UTF-8 and/or UTF-16 and/or UTF-32 support - is available -</pre> -If an unknown option is given, an error message is output; the exit code is 0. -</P> -<P> -<b>-d</b> -Behave as if each pattern has the <b>/D</b> (debug) modifier; the internal -form and information about the compiled pattern is output after compilation; -<b>-d</b> is equivalent to <b>-b -i</b>. -</P> -<P> -<b>-dfa</b> -Behave as if each data line contains the \D escape sequence; this causes the -alternative matching function, <b>pcre[16|32]_dfa_exec()</b>, to be used instead -of the standard <b>pcre[16|32]_exec()</b> function (more detail is given below). -</P> -<P> -<b>-help</b> -Output a brief summary these options and then exit. -</P> -<P> -<b>-i</b> -Behave as if each pattern has the <b>/I</b> modifier; information about the -compiled pattern is given after compilation. -</P> -<P> -<b>-M</b> -Behave as if each data line contains the \M escape sequence; this causes -PCRE to discover the minimum MATCH_LIMIT and MATCH_LIMIT_RECURSION settings by -calling <b>pcre[16|32]_exec()</b> repeatedly with different limits. -</P> -<P> -<b>-m</b> -Output the size of each compiled pattern after it has been compiled. This is -equivalent to adding <b>/M</b> to each regular expression. The size is given in -bytes for both libraries. -</P> -<P> -<b>-O</b> -Behave as if each pattern has the <b>/O</b> modifier, that is disable -auto-possessification for all patterns. -</P> -<P> -<b>-o</b> <i>osize</i> -Set the number of elements in the output vector that is used when calling -<b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> to be <i>osize</i>. The -default value is 45, which is enough for 14 capturing subexpressions for -<b>pcre[16|32]_exec()</b> or 22 different matches for -<b>pcre[16|32]_dfa_exec()</b>. -The vector size can be changed for individual matching calls by including \O -in the data line (see below). -</P> -<P> -<b>-p</b> -Behave as if each pattern has the <b>/P</b> modifier; the POSIX wrapper API is -used to call PCRE. None of the other options has any effect when <b>-p</b> is -set. This option can be used only with the 8-bit library. -</P> -<P> -<b>-q</b> -Do not output the version number of <b>pcretest</b> at the start of execution. -</P> -<P> -<b>-S</b> <i>size</i> -On Unix-like systems, set the size of the run-time stack to <i>size</i> -megabytes. -</P> -<P> -<b>-s</b> or <b>-s+</b> -Behave as if each pattern has the <b>/S</b> modifier; in other words, force each -pattern to be studied. If <b>-s+</b> is used, all the JIT compile options are -passed to <b>pcre[16|32]_study()</b>, causing just-in-time optimization to be set -up if it is available, for both full and partial matching. Specific JIT compile -options can be selected by following <b>-s+</b> with a digit in the range 1 to -7, which selects the JIT compile modes as follows: -<pre> - 1 normal match only - 2 soft partial match only - 3 normal match and soft partial match - 4 hard partial match only - 6 soft and hard partial match - 7 all three modes (default) -</pre> -If <b>-s++</b> is used instead of <b>-s+</b> (with or without a following digit), -the text "(JIT)" is added to the first output line after a match or no match -when JIT-compiled code was actually used. -<br> -<br> -Note that there are pattern options that can override <b>-s</b>, either -specifying no studying at all, or suppressing JIT compilation. -<br> -<br> -If the <b>/I</b> or <b>/D</b> option is present on a pattern (requesting output -about the compiled pattern), information about the result of studying is not -included when studying is caused only by <b>-s</b> and neither <b>-i</b> nor -<b>-d</b> is present on the command line. This behaviour means that the output -from tests that are run with and without <b>-s</b> should be identical, except -when options that output information about the actual running of a match are -set. -<br> -<br> -The <b>-M</b>, <b>-t</b>, and <b>-tm</b> options, which give information about -resources used, are likely to produce different output with and without -<b>-s</b>. Output may also differ if the <b>/C</b> option is present on an -individual pattern. This uses callouts to trace the the matching process, and -this may be different between studied and non-studied patterns. If the pattern -contains (*MARK) items there may also be differences, for the same reason. The -<b>-s</b> command line option can be overridden for specific patterns that -should never be studied (see the <b>/S</b> pattern modifier below). -</P> -<P> -<b>-t</b> -Run each compile, study, and match many times with a timer, and output the -resulting times per compile, study, or match (in milliseconds). Do not set -<b>-m</b> with <b>-t</b>, because you will then get the size output a zillion -times, and the timing will be distorted. You can control the number of -iterations that are used for timing by following <b>-t</b> with a number (as a -separate item on the command line). For example, "-t 1000" iterates 1000 times. -The default is to iterate 500000 times. -</P> -<P> -<b>-tm</b> -This is like <b>-t</b> except that it times only the matching phase, not the -compile or study phases. -</P> -<P> -<b>-T</b> <b>-TM</b> -These behave like <b>-t</b> and <b>-tm</b>, but in addition, at the end of a run, -the total times for all compiles, studies, and matches are output. -</P> -<br><a name="SEC5" href="#TOC1">DESCRIPTION</a><br> -<P> -If <b>pcretest</b> is given two filename arguments, it reads from the first and -writes to the second. If it is given only one filename argument, it reads from -that file and writes to stdout. Otherwise, it reads from stdin and writes to -stdout, and prompts for each line of input, using "re>" to prompt for regular -expressions, and "data>" to prompt for data lines. -</P> -<P> -When <b>pcretest</b> is built, a configuration option can specify that it should -be linked with the <b>libreadline</b> library. When this is done, if the input -is from a terminal, it is read using the <b>readline()</b> function. This -provides line-editing and history facilities. The output from the <b>-help</b> -option states whether or not <b>readline()</b> will be used. -</P> -<P> -The program handles any number of sets of input on a single input file. Each -set starts with a regular expression, and continues with any number of data -lines to be matched against that pattern. -</P> -<P> -Each data line is matched separately and independently. If you want to do -multi-line matches, you have to use the \n escape sequence (or \r or \r\n, -etc., depending on the newline setting) in a single line of input to encode the -newline sequences. There is no limit on the length of data lines; the input -buffer is automatically extended if it is too small. -</P> -<P> -An empty line signals the end of the data lines, at which point a new regular -expression is read. The regular expressions are given enclosed in any -non-alphanumeric delimiters other than backslash, for example: -<pre> - /(a|bc)x+yz/ -</pre> -White space before the initial delimiter is ignored. A regular expression may -be continued over several input lines, in which case the newline characters are -included within it. It is possible to include the delimiter within the pattern -by escaping it, for example -<pre> - /abc\/def/ -</pre> -If you do so, the escape and the delimiter form part of the pattern, but since -delimiters are always non-alphanumeric, this does not affect its interpretation. -If the terminating delimiter is immediately followed by a backslash, for -example, -<pre> - /abc/\ -</pre> -then a backslash is added to the end of the pattern. This is done to provide a -way of testing the error condition that arises if a pattern finishes with a -backslash, because -<pre> - /abc\/ -</pre> -is interpreted as the first line of a pattern that starts with "abc/", causing -pcretest to read the next line as a continuation of the regular expression. -</P> -<br><a name="SEC6" href="#TOC1">PATTERN MODIFIERS</a><br> -<P> -A pattern may be followed by any number of modifiers, which are mostly single -characters, though some of these can be qualified by further characters. -Following Perl usage, these are referred to below as, for example, "the -<b>/i</b> modifier", even though the delimiter of the pattern need not always be -a slash, and no slash is used when writing modifiers. White space may appear -between the final pattern delimiter and the first modifier, and between the -modifiers themselves. For reference, here is a complete list of modifiers. They -fall into several groups that are described in detail in the following -sections. -<pre> - <b>/8</b> set UTF mode - <b>/9</b> set PCRE_NEVER_UTF (locks out UTF mode) - <b>/?</b> disable UTF validity check - <b>/+</b> show remainder of subject after match - <b>/=</b> show all captures (not just those that are set) - - <b>/A</b> set PCRE_ANCHORED - <b>/B</b> show compiled code - <b>/C</b> set PCRE_AUTO_CALLOUT - <b>/D</b> same as <b>/B</b> plus <b>/I</b> - <b>/E</b> set PCRE_DOLLAR_ENDONLY - <b>/F</b> flip byte order in compiled pattern - <b>/f</b> set PCRE_FIRSTLINE - <b>/G</b> find all matches (shorten string) - <b>/g</b> find all matches (use startoffset) - <b>/I</b> show information about pattern - <b>/i</b> set PCRE_CASELESS - <b>/J</b> set PCRE_DUPNAMES - <b>/K</b> show backtracking control names - <b>/L</b> set locale - <b>/M</b> show compiled memory size - <b>/m</b> set PCRE_MULTILINE - <b>/N</b> set PCRE_NO_AUTO_CAPTURE - <b>/O</b> set PCRE_NO_AUTO_POSSESS - <b>/P</b> use the POSIX wrapper - <b>/Q</b> test external stack check function - <b>/S</b> study the pattern after compilation - <b>/s</b> set PCRE_DOTALL - <b>/T</b> select character tables - <b>/U</b> set PCRE_UNGREEDY - <b>/W</b> set PCRE_UCP - <b>/X</b> set PCRE_EXTRA - <b>/x</b> set PCRE_EXTENDED - <b>/Y</b> set PCRE_NO_START_OPTIMIZE - <b>/Z</b> don't show lengths in <b>/B</b> output - - <b>/<any></b> set PCRE_NEWLINE_ANY - <b>/<anycrlf></b> set PCRE_NEWLINE_ANYCRLF - <b>/<cr></b> set PCRE_NEWLINE_CR - <b>/<crlf></b> set PCRE_NEWLINE_CRLF - <b>/<lf></b> set PCRE_NEWLINE_LF - <b>/<bsr_anycrlf></b> set PCRE_BSR_ANYCRLF - <b>/<bsr_unicode></b> set PCRE_BSR_UNICODE - <b>/<JS></b> set PCRE_JAVASCRIPT_COMPAT - -</PRE> -</P> -<br><b> -Perl-compatible modifiers -</b><br> -<P> -The <b>/i</b>, <b>/m</b>, <b>/s</b>, and <b>/x</b> modifiers set the PCRE_CASELESS, -PCRE_MULTILINE, PCRE_DOTALL, or PCRE_EXTENDED options, respectively, when -<b>pcre[16|32]_compile()</b> is called. These four modifier letters have the same -effect as they do in Perl. For example: -<pre> - /caseless/i - -</PRE> -</P> -<br><b> -Modifiers for other PCRE options -</b><br> -<P> -The following table shows additional modifiers for setting PCRE compile-time -options that do not correspond to anything in Perl: -<pre> - <b>/8</b> PCRE_UTF8 ) when using the 8-bit - <b>/?</b> PCRE_NO_UTF8_CHECK ) library - - <b>/8</b> PCRE_UTF16 ) when using the 16-bit - <b>/?</b> PCRE_NO_UTF16_CHECK ) library - - <b>/8</b> PCRE_UTF32 ) when using the 32-bit - <b>/?</b> PCRE_NO_UTF32_CHECK ) library - - <b>/9</b> PCRE_NEVER_UTF - <b>/A</b> PCRE_ANCHORED - <b>/C</b> PCRE_AUTO_CALLOUT - <b>/E</b> PCRE_DOLLAR_ENDONLY - <b>/f</b> PCRE_FIRSTLINE - <b>/J</b> PCRE_DUPNAMES - <b>/N</b> PCRE_NO_AUTO_CAPTURE - <b>/O</b> PCRE_NO_AUTO_POSSESS - <b>/U</b> PCRE_UNGREEDY - <b>/W</b> PCRE_UCP - <b>/X</b> PCRE_EXTRA - <b>/Y</b> PCRE_NO_START_OPTIMIZE - <b>/<any></b> PCRE_NEWLINE_ANY - <b>/<anycrlf></b> PCRE_NEWLINE_ANYCRLF - <b>/<cr></b> PCRE_NEWLINE_CR - <b>/<crlf></b> PCRE_NEWLINE_CRLF - <b>/<lf></b> PCRE_NEWLINE_LF - <b>/<bsr_anycrlf></b> PCRE_BSR_ANYCRLF - <b>/<bsr_unicode></b> PCRE_BSR_UNICODE - <b>/<JS></b> PCRE_JAVASCRIPT_COMPAT -</pre> -The modifiers that are enclosed in angle brackets are literal strings as shown, -including the angle brackets, but the letters within can be in either case. -This example sets multiline matching with CRLF as the line ending sequence: -<pre> - /^abc/m<CRLF> -</pre> -As well as turning on the PCRE_UTF8/16/32 option, the <b>/8</b> modifier causes -all non-printing characters in output strings to be printed using the -\x{hh...} notation. Otherwise, those less than 0x100 are output in hex without -the curly brackets. -</P> -<P> -Full details of the PCRE options are given in the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation. -</P> -<br><b> -Finding all matches in a string -</b><br> -<P> -Searching for all possible matches within each subject string can be requested -by the <b>/g</b> or <b>/G</b> modifier. After finding a match, PCRE is called -again to search the remainder of the subject string. The difference between -<b>/g</b> and <b>/G</b> is that the former uses the <i>startoffset</i> argument to -<b>pcre[16|32]_exec()</b> to start searching at a new point within the entire -string (which is in effect what Perl does), whereas the latter passes over a -shortened substring. This makes a difference to the matching process if the -pattern begins with a lookbehind assertion (including \b or \B). -</P> -<P> -If any call to <b>pcre[16|32]_exec()</b> in a <b>/g</b> or <b>/G</b> sequence matches -an empty string, the next call is done with the PCRE_NOTEMPTY_ATSTART and -PCRE_ANCHORED flags set in order to search for another, non-empty, match at the -same point. If this second match fails, the start offset is advanced, and the -normal match is retried. This imitates the way Perl handles such cases when -using the <b>/g</b> modifier or the <b>split()</b> function. Normally, the start -offset is advanced by one character, but if the newline convention recognizes -CRLF as a newline, and the current character is CR followed by LF, an advance -of two is used. -</P> -<br><b> -Other modifiers -</b><br> -<P> -There are yet more modifiers for controlling the way <b>pcretest</b> -operates. -</P> -<P> -The <b>/+</b> modifier requests that as well as outputting the substring that -matched the entire pattern, <b>pcretest</b> should in addition output the -remainder of the subject string. This is useful for tests where the subject -contains multiple copies of the same substring. If the <b>+</b> modifier appears -twice, the same action is taken for captured substrings. In each case the -remainder is output on the following line with a plus character following the -capture number. Note that this modifier must not immediately follow the /S -modifier because /S+ and /S++ have other meanings. -</P> -<P> -The <b>/=</b> modifier requests that the values of all potential captured -parentheses be output after a match. By default, only those up to the highest -one actually used in the match are output (corresponding to the return code -from <b>pcre[16|32]_exec()</b>). Values in the offsets vector corresponding to -higher numbers should be set to -1, and these are output as "<unset>". This -modifier gives a way of checking that this is happening. -</P> -<P> -The <b>/B</b> modifier is a debugging feature. It requests that <b>pcretest</b> -output a representation of the compiled code after compilation. Normally this -information contains length and offset values; however, if <b>/Z</b> is also -present, this data is replaced by spaces. This is a special feature for use in -the automatic test scripts; it ensures that the same output is generated for -different internal link sizes. -</P> -<P> -The <b>/D</b> modifier is a PCRE debugging feature, and is equivalent to -<b>/BI</b>, that is, both the <b>/B</b> and the <b>/I</b> modifiers. -</P> -<P> -The <b>/F</b> modifier causes <b>pcretest</b> to flip the byte order of the -2-byte and 4-byte fields in the compiled pattern. This facility is for testing -the feature in PCRE that allows it to execute patterns that were compiled on a -host with a different endianness. This feature is not available when the POSIX -interface to PCRE is being used, that is, when the <b>/P</b> pattern modifier is -specified. See also the section about saving and reloading compiled patterns -below. -</P> -<P> -The <b>/I</b> modifier requests that <b>pcretest</b> output information about the -compiled pattern (whether it is anchored, has a fixed first character, and -so on). It does this by calling <b>pcre[16|32]_fullinfo()</b> after compiling a -pattern. If the pattern is studied, the results of that are also output. In -this output, the word "char" means a non-UTF character, that is, the value of a -single data item (8-bit, 16-bit, or 32-bit, depending on the library that is -being tested). -</P> -<P> -The <b>/K</b> modifier requests <b>pcretest</b> to show names from backtracking -control verbs that are returned from calls to <b>pcre[16|32]_exec()</b>. It causes -<b>pcretest</b> to create a <b>pcre[16|32]_extra</b> block if one has not already -been created by a call to <b>pcre[16|32]_study()</b>, and to set the -PCRE_EXTRA_MARK flag and the <b>mark</b> field within it, every time that -<b>pcre[16|32]_exec()</b> is called. If the variable that the <b>mark</b> field -points to is non-NULL for a match, non-match, or partial match, <b>pcretest</b> -prints the string to which it points. For a match, this is shown on a line by -itself, tagged with "MK:". For a non-match it is added to the message. -</P> -<P> -The <b>/L</b> modifier must be followed directly by the name of a locale, for -example, -<pre> - /pattern/Lfr_FR -</pre> -For this reason, it must be the last modifier. The given locale is set, -<b>pcre[16|32]_maketables()</b> is called to build a set of character tables for -the locale, and this is then passed to <b>pcre[16|32]_compile()</b> when compiling -the regular expression. Without an <b>/L</b> (or <b>/T</b>) modifier, NULL is -passed as the tables pointer; that is, <b>/L</b> applies only to the expression -on which it appears. -</P> -<P> -The <b>/M</b> modifier causes the size in bytes of the memory block used to hold -the compiled pattern to be output. This does not include the size of the -<b>pcre[16|32]</b> block; it is just the actual compiled data. If the pattern is -successfully studied with the PCRE_STUDY_JIT_COMPILE option, the size of the -JIT compiled code is also output. -</P> -<P> -The <b>/Q</b> modifier is used to test the use of <b>pcre_stack_guard</b>. It -must be followed by '0' or '1', specifying the return code to be given from an -external function that is passed to PCRE and used for stack checking during -compilation (see the -<a href="pcreapi.html"><b>pcreapi</b></a> -documentation for details). -</P> -<P> -The <b>/S</b> modifier causes <b>pcre[16|32]_study()</b> to be called after the -expression has been compiled, and the results used when the expression is -matched. There are a number of qualifying characters that may follow <b>/S</b>. -They may appear in any order. -</P> -<P> -If <b>/S</b> is followed by an exclamation mark, <b>pcre[16|32]_study()</b> is -called with the PCRE_STUDY_EXTRA_NEEDED option, causing it always to return a -<b>pcre_extra</b> block, even when studying discovers no useful information. -</P> -<P> -If <b>/S</b> is followed by a second S character, it suppresses studying, even -if it was requested externally by the <b>-s</b> command line option. This makes -it possible to specify that certain patterns are always studied, and others are -never studied, independently of <b>-s</b>. This feature is used in the test -files in a few cases where the output is different when the pattern is studied. -</P> -<P> -If the <b>/S</b> modifier is followed by a + character, the call to -<b>pcre[16|32]_study()</b> is made with all the JIT study options, requesting -just-in-time optimization support if it is available, for both normal and -partial matching. If you want to restrict the JIT compiling modes, you can -follow <b>/S+</b> with a digit in the range 1 to 7: -<pre> - 1 normal match only - 2 soft partial match only - 3 normal match and soft partial match - 4 hard partial match only - 6 soft and hard partial match - 7 all three modes (default) -</pre> -If <b>/S++</b> is used instead of <b>/S+</b> (with or without a following digit), -the text "(JIT)" is added to the first output line after a match or no match -when JIT-compiled code was actually used. -</P> -<P> -Note that there is also an independent <b>/+</b> modifier; it must not be given -immediately after <b>/S</b> or <b>/S+</b> because this will be misinterpreted. -</P> -<P> -If JIT studying is successful, the compiled JIT code will automatically be used -when <b>pcre[16|32]_exec()</b> is run, except when incompatible run-time options -are specified. For more details, see the -<a href="pcrejit.html"><b>pcrejit</b></a> -documentation. See also the <b>\J</b> escape sequence below for a way of -setting the size of the JIT stack. -</P> -<P> -Finally, if <b>/S</b> is followed by a minus character, JIT compilation is -suppressed, even if it was requested externally by the <b>-s</b> command line -option. This makes it possible to specify that JIT is never to be used for -certain patterns. -</P> -<P> -The <b>/T</b> modifier must be followed by a single digit. It causes a specific -set of built-in character tables to be passed to <b>pcre[16|32]_compile()</b>. It -is used in the standard PCRE tests to check behaviour with different character -tables. The digit specifies the tables as follows: -<pre> - 0 the default ASCII tables, as distributed in - pcre_chartables.c.dist - 1 a set of tables defining ISO 8859 characters -</pre> -In table 1, some characters whose codes are greater than 128 are identified as -letters, digits, spaces, etc. -</P> -<br><b> -Using the POSIX wrapper API -</b><br> -<P> -The <b>/P</b> modifier causes <b>pcretest</b> to call PCRE via the POSIX wrapper -API rather than its native API. This supports only the 8-bit library. When -<b>/P</b> is set, the following modifiers set options for the <b>regcomp()</b> -function: -<pre> - /i REG_ICASE - /m REG_NEWLINE - /N REG_NOSUB - /s REG_DOTALL ) - /U REG_UNGREEDY ) These options are not part of - /W REG_UCP ) the POSIX standard - /8 REG_UTF8 ) -</pre> -The <b>/+</b> modifier works as described above. All other modifiers are -ignored. -</P> -<br><b> -Locking out certain modifiers -</b><br> -<P> -PCRE can be compiled with or without support for certain features such as -UTF-8/16/32 or Unicode properties. Accordingly, the standard tests are split up -into a number of different files that are selected for running depending on -which features are available. When updating the tests, it is all too easy to -put a new test into the wrong file by mistake; for example, to put a test that -requires UTF support into a file that is used when it is not available. To help -detect such mistakes as early as possible, there is a facility for locking out -specific modifiers. If an input line for <b>pcretest</b> starts with the string -"< forbid " the following sequence of characters is taken as a list of -forbidden modifiers. For example, in the test files that must not use UTF or -Unicode property support, this line appears: -<pre> - < forbid 8W -</pre> -This locks out the /8 and /W modifiers. An immediate error is given if they are -subsequently encountered. If the character string contains < but not >, all the -multi-character modifiers that begin with < are locked out. Otherwise, such -modifiers must be explicitly listed, for example: -<pre> - < forbid <JS><cr> -</pre> -There must be a single space between < and "forbid" for this feature to be -recognised. If there is not, the line is interpreted either as a request to -re-load a pre-compiled pattern (see "SAVING AND RELOADING COMPILED PATTERNS" -below) or, if there is a another < character, as a pattern that uses < as its -delimiter. -</P> -<br><a name="SEC7" href="#TOC1">DATA LINES</a><br> -<P> -Before each data line is passed to <b>pcre[16|32]_exec()</b>, leading and trailing -white space is removed, and it is then scanned for \ escapes. Some of these -are pretty esoteric features, intended for checking out some of the more -complicated features of PCRE. If you are just testing "ordinary" regular -expressions, you probably don't need any of these. The following escapes are -recognized: -<pre> - \a alarm (BEL, \x07) - \b backspace (\x08) - \e escape (\x27) - \f form feed (\x0c) - \n newline (\x0a) - \qdd set the PCRE_MATCH_LIMIT limit to dd (any number of digits) - \r carriage return (\x0d) - \t tab (\x09) - \v vertical tab (\x0b) - \nnn octal character (up to 3 octal digits); always - a byte unless > 255 in UTF-8 or 16-bit or 32-bit mode - \o{dd...} octal character (any number of octal digits} - \xhh hexadecimal byte (up to 2 hex digits) - \x{hh...} hexadecimal character (any number of hex digits) - \A pass the PCRE_ANCHORED option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> - \B pass the PCRE_NOTBOL option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> - \Cdd call pcre[16|32]_copy_substring() for substring dd after a successful match (number less than 32) - \Cname call pcre[16|32]_copy_named_substring() for substring "name" after a successful match (name termin- - ated by next non alphanumeric character) - \C+ show the current captured substrings at callout time - \C- do not supply a callout function - \C!n return 1 instead of 0 when callout number n is reached - \C!n!m return 1 instead of 0 when callout number n is reached for the nth time - \C*n pass the number n (may be negative) as callout data; this is used as the callout return value - \D use the <b>pcre[16|32]_dfa_exec()</b> match function - \F only shortest match for <b>pcre[16|32]_dfa_exec()</b> - \Gdd call pcre[16|32]_get_substring() for substring dd after a successful match (number less than 32) - \Gname call pcre[16|32]_get_named_substring() for substring "name" after a successful match (name termin- - ated by next non-alphanumeric character) - \Jdd set up a JIT stack of dd kilobytes maximum (any number of digits) - \L call pcre[16|32]_get_substringlist() after a successful match - \M discover the minimum MATCH_LIMIT and MATCH_LIMIT_RECURSION settings - \N pass the PCRE_NOTEMPTY option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>; if used twice, pass the - PCRE_NOTEMPTY_ATSTART option - \Odd set the size of the output vector passed to <b>pcre[16|32]_exec()</b> to dd (any number of digits) - \P pass the PCRE_PARTIAL_SOFT option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>; if used twice, pass the - PCRE_PARTIAL_HARD option - \Qdd set the PCRE_MATCH_LIMIT_RECURSION limit to dd (any number of digits) - \R pass the PCRE_DFA_RESTART option to <b>pcre[16|32]_dfa_exec()</b> - \S output details of memory get/free calls during matching - \Y pass the PCRE_NO_START_OPTIMIZE option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> - \Z pass the PCRE_NOTEOL option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> - \? pass the PCRE_NO_UTF[8|16|32]_CHECK option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> - \>dd start the match at offset dd (optional "-"; then any number of digits); this sets the <i>startoffset</i> - argument for <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> - \<cr> pass the PCRE_NEWLINE_CR option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> - \<lf> pass the PCRE_NEWLINE_LF option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> - \<crlf> pass the PCRE_NEWLINE_CRLF option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> - \<anycrlf> pass the PCRE_NEWLINE_ANYCRLF option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> - \<any> pass the PCRE_NEWLINE_ANY option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> -</pre> -The use of \x{hh...} is not dependent on the use of the <b>/8</b> modifier on -the pattern. It is recognized always. There may be any number of hexadecimal -digits inside the braces; invalid values provoke error messages. -</P> -<P> -Note that \xhh specifies one byte rather than one character in UTF-8 mode; -this makes it possible to construct invalid UTF-8 sequences for testing -purposes. On the other hand, \x{hh} is interpreted as a UTF-8 character in -UTF-8 mode, generating more than one byte if the value is greater than 127. -When testing the 8-bit library not in UTF-8 mode, \x{hh} generates one byte -for values less than 256, and causes an error for greater values. -</P> -<P> -In UTF-16 mode, all 4-digit \x{hhhh} values are accepted. This makes it -possible to construct invalid UTF-16 sequences for testing purposes. -</P> -<P> -In UTF-32 mode, all 4- to 8-digit \x{...} values are accepted. This makes it -possible to construct invalid UTF-32 sequences for testing purposes. -</P> -<P> -The escapes that specify line ending sequences are literal strings, exactly as -shown. No more than one newline setting should be present in any data line. -</P> -<P> -A backslash followed by anything else just escapes the anything else. If -the very last character is a backslash, it is ignored. This gives a way of -passing an empty line as data, since a real empty line terminates the data -input. -</P> -<P> -The <b>\J</b> escape provides a way of setting the maximum stack size that is -used by the just-in-time optimization code. It is ignored if JIT optimization -is not being used. Providing a stack that is larger than the default 32K is -necessary only for very complicated patterns. -</P> -<P> -If \M is present, <b>pcretest</b> calls <b>pcre[16|32]_exec()</b> several times, -with different values in the <i>match_limit</i> and <i>match_limit_recursion</i> -fields of the <b>pcre[16|32]_extra</b> data structure, until it finds the minimum -numbers for each parameter that allow <b>pcre[16|32]_exec()</b> to complete without -error. Because this is testing a specific feature of the normal interpretive -<b>pcre[16|32]_exec()</b> execution, the use of any JIT optimization that might -have been set up by the <b>/S+</b> qualifier of <b>-s+</b> option is disabled. -</P> -<P> -The <i>match_limit</i> number is a measure of the amount of backtracking -that takes place, and checking it out can be instructive. For most simple -matches, the number is quite small, but for patterns with very large numbers of -matching possibilities, it can become large very quickly with increasing length -of subject string. The <i>match_limit_recursion</i> number is a measure of how -much stack (or, if PCRE is compiled with NO_RECURSE, how much heap) memory is -needed to complete the match attempt. -</P> -<P> -When \O is used, the value specified may be higher or lower than the size set -by the <b>-O</b> command line option (or defaulted to 45); \O applies only to -the call of <b>pcre[16|32]_exec()</b> for the line in which it appears. -</P> -<P> -If the <b>/P</b> modifier was present on the pattern, causing the POSIX wrapper -API to be used, the only option-setting sequences that have any effect are \B, -\N, and \Z, causing REG_NOTBOL, REG_NOTEMPTY, and REG_NOTEOL, respectively, -to be passed to <b>regexec()</b>. -</P> -<br><a name="SEC8" href="#TOC1">THE ALTERNATIVE MATCHING FUNCTION</a><br> -<P> -By default, <b>pcretest</b> uses the standard PCRE matching function, -<b>pcre[16|32]_exec()</b> to match each data line. PCRE also supports an -alternative matching function, <b>pcre[16|32]_dfa_test()</b>, which operates in a -different way, and has some restrictions. The differences between the two -functions are described in the -<a href="pcrematching.html"><b>pcrematching</b></a> -documentation. -</P> -<P> -If a data line contains the \D escape sequence, or if the command line -contains the <b>-dfa</b> option, the alternative matching function is used. -This function finds all possible matches at a given point. If, however, the \F -escape sequence is present in the data line, it stops after the first match is -found. This is always the shortest possible match. -</P> -<br><a name="SEC9" href="#TOC1">DEFAULT OUTPUT FROM PCRETEST</a><br> -<P> -This section describes the output when the normal matching function, -<b>pcre[16|32]_exec()</b>, is being used. -</P> -<P> -When a match succeeds, <b>pcretest</b> outputs the list of captured substrings -that <b>pcre[16|32]_exec()</b> returns, starting with number 0 for the string that -matched the whole pattern. Otherwise, it outputs "No match" when the return is -PCRE_ERROR_NOMATCH, and "Partial match:" followed by the partially matching -substring when <b>pcre[16|32]_exec()</b> returns PCRE_ERROR_PARTIAL. (Note that -this is the entire substring that was inspected during the partial match; it -may include characters before the actual match start if a lookbehind assertion, -\K, \b, or \B was involved.) For any other return, <b>pcretest</b> outputs -the PCRE negative error number and a short descriptive phrase. If the error is -a failed UTF string check, the offset of the start of the failing character and -the reason code are also output, provided that the size of the output vector is -at least two. Here is an example of an interactive <b>pcretest</b> run. -<pre> - $ pcretest - PCRE version 8.13 2011-04-30 - - re> /^abc(\d+)/ - data> abc123 - 0: abc123 - 1: 123 - data> xyz - No match -</pre> -Unset capturing substrings that are not followed by one that is set are not -returned by <b>pcre[16|32]_exec()</b>, and are not shown by <b>pcretest</b>. In the -following example, there are two capturing substrings, but when the first data -line is matched, the second, unset substring is not shown. An "internal" unset -substring is shown as "<unset>", as for the second data line. -<pre> - re> /(a)|(b)/ - data> a - 0: a - 1: a - data> b - 0: b - 1: <unset> - 2: b -</pre> -If the strings contain any non-printing characters, they are output as \xhh -escapes if the value is less than 256 and UTF mode is not set. Otherwise they -are output as \x{hh...} escapes. See below for the definition of non-printing -characters. If the pattern has the <b>/+</b> modifier, the output for substring -0 is followed by the the rest of the subject string, identified by "0+" like -this: -<pre> - re> /cat/+ - data> cataract - 0: cat - 0+ aract -</pre> -If the pattern has the <b>/g</b> or <b>/G</b> modifier, the results of successive -matching attempts are output in sequence, like this: -<pre> - re> /\Bi(\w\w)/g - data> Mississippi - 0: iss - 1: ss - 0: iss - 1: ss - 0: ipp - 1: pp -</pre> -"No match" is output only if the first match attempt fails. Here is an example -of a failure message (the offset 4 that is specified by \>4 is past the end of -the subject string): -<pre> - re> /xyz/ - data> xyz\>4 - Error -24 (bad offset value) -</PRE> -</P> -<P> -If any of the sequences <b>\C</b>, <b>\G</b>, or <b>\L</b> are present in a -data line that is successfully matched, the substrings extracted by the -convenience functions are output with C, G, or L after the string number -instead of a colon. This is in addition to the normal full list. The string -length (that is, the return from the extraction function) is given in -parentheses after each string for <b>\C</b> and <b>\G</b>. -</P> -<P> -Note that whereas patterns can be continued over several lines (a plain ">" -prompt is used for continuations), data lines may not. However newlines can be -included in data by means of the \n escape (or \r, \r\n, etc., depending on -the newline sequence setting). -</P> -<br><a name="SEC10" href="#TOC1">OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION</a><br> -<P> -When the alternative matching function, <b>pcre[16|32]_dfa_exec()</b>, is used (by -means of the \D escape sequence or the <b>-dfa</b> command line option), the -output consists of a list of all the matches that start at the first point in -the subject where there is at least one match. For example: -<pre> - re> /(tang|tangerine|tan)/ - data> yellow tangerine\D - 0: tangerine - 1: tang - 2: tan -</pre> -(Using the normal matching function on this data finds only "tang".) The -longest matching string is always given first (and numbered zero). After a -PCRE_ERROR_PARTIAL return, the output is "Partial match:", followed by the -partially matching substring. (Note that this is the entire substring that was -inspected during the partial match; it may include characters before the actual -match start if a lookbehind assertion, \K, \b, or \B was involved.) -</P> -<P> -If <b>/g</b> is present on the pattern, the search for further matches resumes -at the end of the longest match. For example: -<pre> - re> /(tang|tangerine|tan)/g - data> yellow tangerine and tangy sultana\D - 0: tangerine - 1: tang - 2: tan - 0: tang - 1: tan - 0: tan -</pre> -Since the matching function does not support substring capture, the escape -sequences that are concerned with captured substrings are not relevant. -</P> -<br><a name="SEC11" href="#TOC1">RESTARTING AFTER A PARTIAL MATCH</a><br> -<P> -When the alternative matching function has given the PCRE_ERROR_PARTIAL return, -indicating that the subject partially matched the pattern, you can restart the -match with additional subject data by means of the \R escape sequence. For -example: -<pre> - re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ - data> 23ja\P\D - Partial match: 23ja - data> n05\R\D - 0: n05 -</pre> -For further information about partial matching, see the -<a href="pcrepartial.html"><b>pcrepartial</b></a> -documentation. -</P> -<br><a name="SEC12" href="#TOC1">CALLOUTS</a><br> -<P> -If the pattern contains any callout requests, <b>pcretest</b>'s callout function -is called during matching. This works with both matching functions. By default, -the called function displays the callout number, the start and current -positions in the text at the callout time, and the next pattern item to be -tested. For example: -<pre> - --->pqrabcdef - 0 ^ ^ \d -</pre> -This output indicates that callout number 0 occurred for a match attempt -starting at the fourth character of the subject string, when the pointer was at -the seventh character of the data, and when the next pattern item was \d. Just -one circumflex is output if the start and current positions are the same. -</P> -<P> -Callouts numbered 255 are assumed to be automatic callouts, inserted as a -result of the <b>/C</b> pattern modifier. In this case, instead of showing the -callout number, the offset in the pattern, preceded by a plus, is output. For -example: -<pre> - re> /\d?[A-E]\*/C - data> E* - --->E* - +0 ^ \d? - +3 ^ [A-E] - +8 ^^ \* - +10 ^ ^ - 0: E* -</pre> -If a pattern contains (*MARK) items, an additional line is output whenever -a change of latest mark is passed to the callout function. For example: -<pre> - re> /a(*MARK:X)bc/C - data> abc - --->abc - +0 ^ a - +1 ^^ (*MARK:X) - +10 ^^ b - Latest Mark: X - +11 ^ ^ c - +12 ^ ^ - 0: abc -</pre> -The mark changes between matching "a" and "b", but stays the same for the rest -of the match, so nothing more is output. If, as a result of backtracking, the -mark reverts to being unset, the text "<unset>" is output. -</P> -<P> -The callout function in <b>pcretest</b> returns zero (carry on matching) by -default, but you can use a \C item in a data line (as described above) to -change this and other parameters of the callout. -</P> -<P> -Inserting callouts can be helpful when using <b>pcretest</b> to check -complicated regular expressions. For further information about callouts, see -the -<a href="pcrecallout.html"><b>pcrecallout</b></a> -documentation. -</P> -<br><a name="SEC13" href="#TOC1">NON-PRINTING CHARACTERS</a><br> -<P> -When <b>pcretest</b> is outputting text in the compiled version of a pattern, -bytes other than 32-126 are always treated as non-printing characters are are -therefore shown as hex escapes. -</P> -<P> -When <b>pcretest</b> is outputting text that is a matched part of a subject -string, it behaves in the same way, unless a different locale has been set for -the pattern (using the <b>/L</b> modifier). In this case, the <b>isprint()</b> -function to distinguish printing and non-printing characters. -</P> -<br><a name="SEC14" href="#TOC1">SAVING AND RELOADING COMPILED PATTERNS</a><br> -<P> -The facilities described in this section are not available when the POSIX -interface to PCRE is being used, that is, when the <b>/P</b> pattern modifier is -specified. -</P> -<P> -When the POSIX interface is not in use, you can cause <b>pcretest</b> to write a -compiled pattern to a file, by following the modifiers with > and a file name. -For example: -<pre> - /pattern/im >/some/file -</pre> -See the -<a href="pcreprecompile.html"><b>pcreprecompile</b></a> -documentation for a discussion about saving and re-using compiled patterns. -Note that if the pattern was successfully studied with JIT optimization, the -JIT data cannot be saved. -</P> -<P> -The data that is written is binary. The first eight bytes are the length of the -compiled pattern data followed by the length of the optional study data, each -written as four bytes in big-endian order (most significant byte first). If -there is no study data (either the pattern was not studied, or studying did not -return any data), the second length is zero. The lengths are followed by an -exact copy of the compiled pattern. If there is additional study data, this -(excluding any JIT data) follows immediately after the compiled pattern. After -writing the file, <b>pcretest</b> expects to read a new pattern. -</P> -<P> -A saved pattern can be reloaded into <b>pcretest</b> by specifying < and a file -name instead of a pattern. There must be no space between < and the file name, -which must not contain a < character, as otherwise <b>pcretest</b> will -interpret the line as a pattern delimited by < characters. For example: -<pre> - re> </some/file - Compiled pattern loaded from /some/file - No study data -</pre> -If the pattern was previously studied with the JIT optimization, the JIT -information cannot be saved and restored, and so is lost. When the pattern has -been loaded, <b>pcretest</b> proceeds to read data lines in the usual way. -</P> -<P> -You can copy a file written by <b>pcretest</b> to a different host and reload it -there, even if the new host has opposite endianness to the one on which the -pattern was compiled. For example, you can compile on an i86 machine and run on -a SPARC machine. When a pattern is reloaded on a host with different -endianness, the confirmation message is changed to: -<pre> - Compiled pattern (byte-inverted) loaded from /some/file -</pre> -The test suite contains some saved pre-compiled patterns with different -endianness. These are reloaded using "<!" instead of just "<". This suppresses -the "(byte-inverted)" text so that the output is the same on all hosts. It also -forces debugging output once the pattern has been reloaded. -</P> -<P> -File names for saving and reloading can be absolute or relative, but note that -the shell facility of expanding a file name that starts with a tilde (~) is not -available. -</P> -<P> -The ability to save and reload files in <b>pcretest</b> is intended for testing -and experimentation. It is not intended for production use because only a -single pattern can be written to a file. Furthermore, there is no facility for -supplying custom character tables for use with a reloaded pattern. If the -original pattern was compiled with custom tables, an attempt to match a subject -string using a reloaded pattern is likely to cause <b>pcretest</b> to crash. -Finally, if you attempt to load a file that is not in the correct format, the -result is undefined. -</P> -<br><a name="SEC15" href="#TOC1">SEE ALSO</a><br> -<P> -<b>pcre</b>(3), <b>pcre16</b>(3), <b>pcre32</b>(3), <b>pcreapi</b>(3), -<b>pcrecallout</b>(3), -<b>pcrejit</b>, <b>pcrematching</b>(3), <b>pcrepartial</b>(d), -<b>pcrepattern</b>(3), <b>pcreprecompile</b>(3). -</P> -<br><a name="SEC16" href="#TOC1">AUTHOR</a><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><a name="SEC17" href="#TOC1">REVISION</a><br> -<P> -Last updated: 09 February 2014 -<br> -Copyright © 1997-2014 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/html/pcreunicode.html b/plugins/Pcre16/docs/doc/html/pcreunicode.html deleted file mode 100644 index ab36bc61e3..0000000000 --- a/plugins/Pcre16/docs/doc/html/pcreunicode.html +++ /dev/null @@ -1,262 +0,0 @@ -<html> -<head> -<title>pcreunicode specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>pcreunicode man page</h1> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> -<p> -This page is part of the PCRE HTML documentation. It was generated automatically -from the original man page. If there is any nonsense in it, please consult the -man page, in case the conversion went wrong. -<br> -<br><b> -UTF-8, UTF-16, UTF-32, AND UNICODE PROPERTY SUPPORT -</b><br> -<P> -As well as UTF-8 support, PCRE also supports UTF-16 (from release 8.30) and -UTF-32 (from release 8.32), by means of two additional libraries. They can be -built as well as, or instead of, the 8-bit library. -</P> -<br><b> -UTF-8 SUPPORT -</b><br> -<P> -In order process UTF-8 strings, you must build PCRE's 8-bit library with UTF -support, and, in addition, you must call -<a href="pcre_compile.html"><b>pcre_compile()</b></a> -with the PCRE_UTF8 option flag, or the pattern must start with the sequence -(*UTF8) or (*UTF). When either of these is the case, both the pattern and any -subject strings that are matched against it are treated as UTF-8 strings -instead of strings of individual 1-byte characters. -</P> -<br><b> -UTF-16 AND UTF-32 SUPPORT -</b><br> -<P> -In order process UTF-16 or UTF-32 strings, you must build PCRE's 16-bit or -32-bit library with UTF support, and, in addition, you must call -<a href="pcre16_compile.html"><b>pcre16_compile()</b></a> -or -<a href="pcre32_compile.html"><b>pcre32_compile()</b></a> -with the PCRE_UTF16 or PCRE_UTF32 option flag, as appropriate. Alternatively, -the pattern must start with the sequence (*UTF16), (*UTF32), as appropriate, or -(*UTF), which can be used with either library. When UTF mode is set, both the -pattern and any subject strings that are matched against it are treated as -UTF-16 or UTF-32 strings instead of strings of individual 16-bit or 32-bit -characters. -</P> -<br><b> -UTF SUPPORT OVERHEAD -</b><br> -<P> -If you compile PCRE with UTF support, but do not use it at run time, the -library will be a bit bigger, but the additional run time overhead is limited -to testing the PCRE_UTF[8|16|32] flag occasionally, so should not be very big. -</P> -<br><b> -UNICODE PROPERTY SUPPORT -</b><br> -<P> -If PCRE is built with Unicode character property support (which implies UTF -support), the escape sequences \p{..}, \P{..}, and \X can be used. -The available properties that can be tested are limited to the general -category properties such as Lu for an upper case letter or Nd for a decimal -number, the Unicode script names such as Arabic or Han, and the derived -properties Any and L&. Full lists is given in the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -and -<a href="pcresyntax.html"><b>pcresyntax</b></a> -documentation. Only the short names for properties are supported. For example, -\p{L} matches a letter. Its Perl synonym, \p{Letter}, is not supported. -Furthermore, in Perl, many properties may optionally be prefixed by "Is", for -compatibility with Perl 5.6. PCRE does not support this. -<a name="utf8strings"></a></P> -<br><b> -Validity of UTF-8 strings -</b><br> -<P> -When you set the PCRE_UTF8 flag, the byte strings passed as patterns and -subjects are (by default) checked for validity on entry to the relevant -functions. The entire string is checked before any other processing takes -place. From release 7.3 of PCRE, the check is according the rules of RFC 3629, -which are themselves derived from the Unicode specification. Earlier releases -of PCRE followed the rules of RFC 2279, which allows the full range of 31-bit -values (0 to 0x7FFFFFFF). The current check allows only values in the range U+0 -to U+10FFFF, excluding the surrogate area. (From release 8.33 the so-called -"non-character" code points are no longer excluded because Unicode corrigendum -#9 makes it clear that they should not be.) -</P> -<P> -Characters in the "Surrogate Area" of Unicode are reserved for use by UTF-16, -where they are used in pairs to encode codepoints with values greater than -0xFFFF. The code points that are encoded by UTF-16 pairs are available -independently in the UTF-8 and UTF-32 encodings. (In other words, the whole -surrogate thing is a fudge for UTF-16 which unfortunately messes up UTF-8 and -UTF-32.) -</P> -<P> -If an invalid UTF-8 string is passed to PCRE, an error return is given. At -compile time, the only additional information is the offset to the first byte -of the failing character. The run-time functions <b>pcre_exec()</b> and -<b>pcre_dfa_exec()</b> also pass back this information, as well as a more -detailed reason code if the caller has provided memory in which to do this. -</P> -<P> -In some situations, you may already know that your strings are valid, and -therefore want to skip these checks in order to improve performance, for -example in the case of a long subject string that is being scanned repeatedly. -If you set the PCRE_NO_UTF8_CHECK flag at compile time or at run time, PCRE -assumes that the pattern or subject it is given (respectively) contains only -valid UTF-8 codes. In this case, it does not diagnose an invalid UTF-8 string. -</P> -<P> -Note that passing PCRE_NO_UTF8_CHECK to <b>pcre_compile()</b> just disables the -check for the pattern; it does not also apply to subject strings. If you want -to disable the check for a subject string you must pass this option to -<b>pcre_exec()</b> or <b>pcre_dfa_exec()</b>. -</P> -<P> -If you pass an invalid UTF-8 string when PCRE_NO_UTF8_CHECK is set, the result -is undefined and your program may crash. -<a name="utf16strings"></a></P> -<br><b> -Validity of UTF-16 strings -</b><br> -<P> -When you set the PCRE_UTF16 flag, the strings of 16-bit data units that are -passed as patterns and subjects are (by default) checked for validity on entry -to the relevant functions. Values other than those in the surrogate range -U+D800 to U+DFFF are independent code points. Values in the surrogate range -must be used in pairs in the correct manner. -</P> -<P> -If an invalid UTF-16 string is passed to PCRE, an error return is given. At -compile time, the only additional information is the offset to the first data -unit of the failing character. The run-time functions <b>pcre16_exec()</b> and -<b>pcre16_dfa_exec()</b> also pass back this information, as well as a more -detailed reason code if the caller has provided memory in which to do this. -</P> -<P> -In some situations, you may already know that your strings are valid, and -therefore want to skip these checks in order to improve performance. If you set -the PCRE_NO_UTF16_CHECK flag at compile time or at run time, PCRE assumes that -the pattern or subject it is given (respectively) contains only valid UTF-16 -sequences. In this case, it does not diagnose an invalid UTF-16 string. -However, if an invalid string is passed, the result is undefined. -<a name="utf32strings"></a></P> -<br><b> -Validity of UTF-32 strings -</b><br> -<P> -When you set the PCRE_UTF32 flag, the strings of 32-bit data units that are -passed as patterns and subjects are (by default) checked for validity on entry -to the relevant functions. This check allows only values in the range U+0 -to U+10FFFF, excluding the surrogate area U+D800 to U+DFFF. -</P> -<P> -If an invalid UTF-32 string is passed to PCRE, an error return is given. At -compile time, the only additional information is the offset to the first data -unit of the failing character. The run-time functions <b>pcre32_exec()</b> and -<b>pcre32_dfa_exec()</b> also pass back this information, as well as a more -detailed reason code if the caller has provided memory in which to do this. -</P> -<P> -In some situations, you may already know that your strings are valid, and -therefore want to skip these checks in order to improve performance. If you set -the PCRE_NO_UTF32_CHECK flag at compile time or at run time, PCRE assumes that -the pattern or subject it is given (respectively) contains only valid UTF-32 -sequences. In this case, it does not diagnose an invalid UTF-32 string. -However, if an invalid string is passed, the result is undefined. -</P> -<br><b> -General comments about UTF modes -</b><br> -<P> -1. Codepoints less than 256 can be specified in patterns by either braced or -unbraced hexadecimal escape sequences (for example, \x{b3} or \xb3). Larger -values have to use braced sequences. -</P> -<P> -2. Octal numbers up to \777 are recognized, and in UTF-8 mode they match -two-byte characters for values greater than \177. -</P> -<P> -3. Repeat quantifiers apply to complete UTF characters, not to individual -data units, for example: \x{100}{3}. -</P> -<P> -4. The dot metacharacter matches one UTF character instead of a single data -unit. -</P> -<P> -5. The escape sequence \C can be used to match a single byte in UTF-8 mode, or -a single 16-bit data unit in UTF-16 mode, or a single 32-bit data unit in -UTF-32 mode, but its use can lead to some strange effects because it breaks up -multi-unit characters (see the description of \C in the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -documentation). The use of \C is not supported in the alternative matching -function <b>pcre[16|32]_dfa_exec()</b>, nor is it supported in UTF mode by the -JIT optimization of <b>pcre[16|32]_exec()</b>. If JIT optimization is requested -for a UTF pattern that contains \C, it will not succeed, and so the matching -will be carried out by the normal interpretive function. -</P> -<P> -6. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly -test characters of any code value, but, by default, the characters that PCRE -recognizes as digits, spaces, or word characters remain the same set as in -non-UTF mode, all with values less than 256. This remains true even when PCRE -is built to include Unicode property support, because to do otherwise would -slow down PCRE in many common cases. Note in particular that this applies to -\b and \B, because they are defined in terms of \w and \W. If you really -want to test for a wider sense of, say, "digit", you can use explicit Unicode -property tests such as \p{Nd}. Alternatively, if you set the PCRE_UCP option, -the way that the character escapes work is changed so that Unicode properties -are used to determine which characters match. There are more details in the -section on -<a href="pcrepattern.html#genericchartypes">generic character types</a> -in the -<a href="pcrepattern.html"><b>pcrepattern</b></a> -documentation. -</P> -<P> -7. Similarly, characters that match the POSIX named character classes are all -low-valued characters, unless the PCRE_UCP option is set. -</P> -<P> -8. However, the horizontal and vertical white space matching escapes (\h, \H, -\v, and \V) do match all the appropriate Unicode characters, whether or not -PCRE_UCP is set. -</P> -<P> -9. Case-insensitive matching applies only to characters whose values are less -than 128, unless PCRE is built with Unicode property support. A few Unicode -characters such as Greek sigma have more than two codepoints that are -case-equivalent. Up to and including PCRE release 8.31, only one-to-one case -mappings were supported, but later releases (with Unicode property support) do -treat as case-equivalent all versions of characters such as Greek sigma. -</P> -<br><b> -AUTHOR -</b><br> -<P> -Philip Hazel -<br> -University Computing Service -<br> -Cambridge CB2 3QH, England. -<br> -</P> -<br><b> -REVISION -</b><br> -<P> -Last updated: 27 February 2013 -<br> -Copyright © 1997-2013 University of Cambridge. -<br> -<p> -Return to the <a href="index.html">PCRE index page</a>. -</p> diff --git a/plugins/Pcre16/docs/doc/index.html.src b/plugins/Pcre16/docs/doc/index.html.src deleted file mode 100644 index 887f4d7507..0000000000 --- a/plugins/Pcre16/docs/doc/index.html.src +++ /dev/null @@ -1,185 +0,0 @@ -<html> -<!-- This is a manually maintained file that is the root of the HTML version of - the PCRE documentation. When the HTML documents are built from the man - page versions, the entire doc/html directory is emptied, this file is then - copied into doc/html/index.html, and the remaining files therein are - created by the 132html script. ---> -<head> -<title>PCRE specification</title> -</head> -<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> -<h1>Perl-compatible Regular Expressions (PCRE)</h1> -<p> -The HTML documentation for PCRE consists of a number of pages that are listed -below in alphabetical order. If you are new to PCRE, please read the first one -first. -</p> - -<table> -<tr><td><a href="pcre.html">pcre</a></td> - <td> Introductory page</td></tr> - -<tr><td><a href="pcre-config.html">pcre-config</a></td> - <td> Information about the installation configuration</td></tr> - -<tr><td><a href="pcre16.html">pcre16</a></td> - <td> Discussion of the 16-bit PCRE library</td></tr> - -<tr><td><a href="pcre32.html">pcre32</a></td> - <td> Discussion of the 32-bit PCRE library</td></tr> - -<tr><td><a href="pcreapi.html">pcreapi</a></td> - <td> PCRE's native API</td></tr> - -<tr><td><a href="pcrebuild.html">pcrebuild</a></td> - <td> Building PCRE</td></tr> - -<tr><td><a href="pcrecallout.html">pcrecallout</a></td> - <td> The <i>callout</i> facility</td></tr> - -<tr><td><a href="pcrecompat.html">pcrecompat</a></td> - <td> Compability with Perl</td></tr> - -<tr><td><a href="pcrecpp.html">pcrecpp</a></td> - <td> The C++ wrapper for the PCRE library</td></tr> - -<tr><td><a href="pcredemo.html">pcredemo</a></td> - <td> A demonstration C program that uses the PCRE library</td></tr> - -<tr><td><a href="pcregrep.html">pcregrep</a></td> - <td> The <b>pcregrep</b> command</td></tr> - -<tr><td><a href="pcrejit.html">pcrejit</a></td> - <td> Discussion of the just-in-time optimization support</td></tr> - -<tr><td><a href="pcrelimits.html">pcrelimits</a></td> - <td> Details of size and other limits</td></tr> - -<tr><td><a href="pcrematching.html">pcrematching</a></td> - <td> Discussion of the two matching algorithms</td></tr> - -<tr><td><a href="pcrepartial.html">pcrepartial</a></td> - <td> Using PCRE for partial matching</td></tr> - -<tr><td><a href="pcrepattern.html">pcrepattern</a></td> - <td> Specification of the regular expressions supported by PCRE</td></tr> - -<tr><td><a href="pcreperform.html">pcreperform</a></td> - <td> Some comments on performance</td></tr> - -<tr><td><a href="pcreposix.html">pcreposix</a></td> - <td> The POSIX API to the PCRE 8-bit library</td></tr> - -<tr><td><a href="pcreprecompile.html">pcreprecompile</a></td> - <td> How to save and re-use compiled patterns</td></tr> - -<tr><td><a href="pcresample.html">pcresample</a></td> - <td> Discussion of the pcredemo program</td></tr> - -<tr><td><a href="pcrestack.html">pcrestack</a></td> - <td> Discussion of PCRE's stack usage</td></tr> - -<tr><td><a href="pcresyntax.html">pcresyntax</a></td> - <td> Syntax quick-reference summary</td></tr> - -<tr><td><a href="pcretest.html">pcretest</a></td> - <td> The <b>pcretest</b> command for testing PCRE</td></tr> - -<tr><td><a href="pcreunicode.html">pcreunicode</a></td> - <td> Discussion of Unicode and UTF-8/UTF-16/UTF-32 support</td></tr> -</table> - -<p> -There are also individual pages that summarize the interface for each function -in the library. There is a single page for each triple of 8-bit/16-bit/32-bit -functions. -</p> - -<table> - -<tr><td><a href="pcre_assign_jit_stack.html">pcre_assign_jit_stack</a></td> - <td> Assign stack for JIT matching</td></tr> - -<tr><td><a href="pcre_compile.html">pcre_compile</a></td> - <td> Compile a regular expression</td></tr> - -<tr><td><a href="pcre_compile2.html">pcre_compile2</a></td> - <td> Compile a regular expression (alternate interface)</td></tr> - -<tr><td><a href="pcre_config.html">pcre_config</a></td> - <td> Show build-time configuration options</td></tr> - -<tr><td><a href="pcre_copy_named_substring.html">pcre_copy_named_substring</a></td> - <td> Extract named substring into given buffer</td></tr> - -<tr><td><a href="pcre_copy_substring.html">pcre_copy_substring</a></td> - <td> Extract numbered substring into given buffer</td></tr> - -<tr><td><a href="pcre_dfa_exec.html">pcre_dfa_exec</a></td> - <td> Match a compiled pattern to a subject string - (DFA algorithm; <i>not</i> Perl compatible)</td></tr> - -<tr><td><a href="pcre_exec.html">pcre_exec</a></td> - <td> Match a compiled pattern to a subject string - (Perl compatible)</td></tr> - -<tr><td><a href="pcre_free_study.html">pcre_free_study</a></td> - <td> Free study data</td></tr> - -<tr><td><a href="pcre_free_substring.html">pcre_free_substring</a></td> - <td> Free extracted substring</td></tr> - -<tr><td><a href="pcre_free_substring_list.html">pcre_free_substring_list</a></td> - <td> Free list of extracted substrings</td></tr> - -<tr><td><a href="pcre_fullinfo.html">pcre_fullinfo</a></td> - <td> Extract information about a pattern</td></tr> - -<tr><td><a href="pcre_get_named_substring.html">pcre_get_named_substring</a></td> - <td> Extract named substring into new memory</td></tr> - -<tr><td><a href="pcre_get_stringnumber.html">pcre_get_stringnumber</a></td> - <td> Convert captured string name to number</td></tr> - -<tr><td><a href="pcre_get_stringtable_entries.html">pcre_get_stringtable_entries</a></td> - <td> Find table entries for given string name</td></tr> - -<tr><td><a href="pcre_get_substring.html">pcre_get_substring</a></td> - <td> Extract numbered substring into new memory</td></tr> - -<tr><td><a href="pcre_get_substring_list.html">pcre_get_substring_list</a></td> - <td> Extract all substrings into new memory</td></tr> - -<tr><td><a href="pcre_jit_exec.html">pcre_jit_exec</a></td> - <td> Fast path interface to JIT matching</td></tr> - -<tr><td><a href="pcre_jit_stack_alloc.html">pcre_jit_stack_alloc</a></td> - <td> Create a stack for JIT matching</td></tr> - -<tr><td><a href="pcre_jit_stack_free.html">pcre_jit_stack_free</a></td> - <td> Free a JIT matching stack</td></tr> - -<tr><td><a href="pcre_maketables.html">pcre_maketables</a></td> - <td> Build character tables in current locale</td></tr> - -<tr><td><a href="pcre_pattern_to_host_byte_order.html">pcre_pattern_to_host_byte_order</a></td> - <td> Convert compiled pattern to host byte order if necessary</td></tr> - -<tr><td><a href="pcre_refcount.html">pcre_refcount</a></td> - <td> Maintain reference count in compiled pattern</td></tr> - -<tr><td><a href="pcre_study.html">pcre_study</a></td> - <td> Study a compiled pattern</td></tr> - -<tr><td><a href="pcre_utf16_to_host_byte_order.html">pcre_utf16_to_host_byte_order</a></td> - <td> Convert UTF-16 string to host byte order if necessary</td></tr> - -<tr><td><a href="pcre_utf32_to_host_byte_order.html">pcre_utf32_to_host_byte_order</a></td> - <td> Convert UTF-32 string to host byte order if necessary</td></tr> - -<tr><td><a href="pcre_version.html">pcre_version</a></td> - <td> Return PCRE version and release date</td></tr> -</table> - -</html> diff --git a/plugins/Pcre16/docs/doc/pcre-config.1 b/plugins/Pcre16/docs/doc/pcre-config.1 deleted file mode 100644 index 52eb4fb226..0000000000 --- a/plugins/Pcre16/docs/doc/pcre-config.1 +++ /dev/null @@ -1,92 +0,0 @@ -.TH PCRE-CONFIG 1 "01 January 2012" "PCRE 8.30" -.SH NAME -pcre-config - program to return PCRE configuration -.SH SYNOPSIS -.rs -.sp -.nf -.B pcre-config [--prefix] [--exec-prefix] [--version] [--libs] -.B " [--libs16] [--libs32] [--libs-cpp] [--libs-posix]" -.B " [--cflags] [--cflags-posix]" -.fi -. -. -.SH DESCRIPTION -.rs -.sp -\fBpcre-config\fP returns the configuration of the installed PCRE -libraries and the options required to compile a program to use them. Some of -the options apply only to the 8-bit, or 16-bit, or 32-bit libraries, -respectively, and are -not available if only one of those libraries has been built. If an unavailable -option is encountered, the "usage" information is output. -. -. -.SH OPTIONS -.rs -.TP 10 -\fB--prefix\fP -Writes the directory prefix used in the PCRE installation for architecture -independent files (\fI/usr\fP on many systems, \fI/usr/local\fP on some -systems) to the standard output. -.TP 10 -\fB--exec-prefix\fP -Writes the directory prefix used in the PCRE installation for architecture -dependent files (normally the same as \fB--prefix\fP) to the standard output. -.TP 10 -\fB--version\fP -Writes the version number of the installed PCRE libraries to the standard -output. -.TP 10 -\fB--libs\fP -Writes to the standard output the command line options required to link -with the 8-bit PCRE library (\fB-lpcre\fP on many systems). -.TP 10 -\fB--libs16\fP -Writes to the standard output the command line options required to link -with the 16-bit PCRE library (\fB-lpcre16\fP on many systems). -.TP 10 -\fB--libs32\fP -Writes to the standard output the command line options required to link -with the 32-bit PCRE library (\fB-lpcre32\fP on many systems). -.TP 10 -\fB--libs-cpp\fP -Writes to the standard output the command line options required to link with -PCRE's C++ wrapper library (\fB-lpcrecpp\fP \fB-lpcre\fP on many -systems). -.TP 10 -\fB--libs-posix\fP -Writes to the standard output the command line options required to link with -PCRE's POSIX API wrapper library (\fB-lpcreposix\fP \fB-lpcre\fP on many -systems). -.TP 10 -\fB--cflags\fP -Writes to the standard output the command line options required to compile -files that use PCRE (this may include some \fB-I\fP options, but is blank on -many systems). -.TP 10 -\fB--cflags-posix\fP -Writes to the standard output the command line options required to compile -files that use PCRE's POSIX API wrapper library (this may include some \fB-I\fP -options, but is blank on many systems). -. -. -.SH "SEE ALSO" -.rs -.sp -\fBpcre(3)\fP -. -. -.SH AUTHOR -.rs -.sp -This manual page was originally written by Mark Baker for the Debian GNU/Linux -system. It has been subsequently revised as a generic PCRE man page. -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 24 June 2012 -.fi diff --git a/plugins/Pcre16/docs/doc/pcre-config.txt b/plugins/Pcre16/docs/doc/pcre-config.txt deleted file mode 100644 index 8503ab0ef0..0000000000 --- a/plugins/Pcre16/docs/doc/pcre-config.txt +++ /dev/null @@ -1,86 +0,0 @@ -PCRE-CONFIG(1) General Commands Manual PCRE-CONFIG(1) - - - -NAME - pcre-config - program to return PCRE configuration - -SYNOPSIS - - pcre-config [--prefix] [--exec-prefix] [--version] [--libs] - [--libs16] [--libs32] [--libs-cpp] [--libs-posix] - [--cflags] [--cflags-posix] - - -DESCRIPTION - - pcre-config returns the configuration of the installed PCRE libraries - and the options required to compile a program to use them. Some of the - options apply only to the 8-bit, or 16-bit, or 32-bit libraries, - respectively, and are not available if only one of those libraries has - been built. If an unavailable option is encountered, the "usage" infor- - mation is output. - - -OPTIONS - - --prefix Writes the directory prefix used in the PCRE installation for - architecture independent files (/usr on many systems, - /usr/local on some systems) to the standard output. - - --exec-prefix - Writes the directory prefix used in the PCRE installation for - architecture dependent files (normally the same as --prefix) - to the standard output. - - --version Writes the version number of the installed PCRE libraries to - the standard output. - - --libs Writes to the standard output the command line options - required to link with the 8-bit PCRE library (-lpcre on many - systems). - - --libs16 Writes to the standard output the command line options - required to link with the 16-bit PCRE library (-lpcre16 on - many systems). - - --libs32 Writes to the standard output the command line options - required to link with the 32-bit PCRE library (-lpcre32 on - many systems). - - --libs-cpp - Writes to the standard output the command line options - required to link with PCRE's C++ wrapper library (-lpcrecpp - -lpcre on many systems). - - --libs-posix - Writes to the standard output the command line options - required to link with PCRE's POSIX API wrapper library - (-lpcreposix -lpcre on many systems). - - --cflags Writes to the standard output the command line options - required to compile files that use PCRE (this may include - some -I options, but is blank on many systems). - - --cflags-posix - Writes to the standard output the command line options - required to compile files that use PCRE's POSIX API wrapper - library (this may include some -I options, but is blank on - many systems). - - -SEE ALSO - - pcre(3) - - -AUTHOR - - This manual page was originally written by Mark Baker for the Debian - GNU/Linux system. It has been subsequently revised as a generic PCRE - man page. - - -REVISION - - Last updated: 24 June 2012 diff --git a/plugins/Pcre16/docs/doc/pcre.3 b/plugins/Pcre16/docs/doc/pcre.3 deleted file mode 100644 index 4eda404ccf..0000000000 --- a/plugins/Pcre16/docs/doc/pcre.3 +++ /dev/null @@ -1,218 +0,0 @@ -.TH PCRE 3 "08 January 2014" "PCRE 8.35" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH INTRODUCTION -.rs -.sp -The PCRE library is a set of functions that implement regular expression -pattern matching using the same syntax and semantics as Perl, with just a few -differences. Some features that appeared in Python and PCRE before they -appeared in Perl are also available using the Python syntax, there is some -support for one or two .NET and Oniguruma syntax items, and there is an option -for requesting some minor changes that give better JavaScript compatibility. -.P -Starting with release 8.30, it is possible to compile two separate PCRE -libraries: the original, which supports 8-bit character strings (including -UTF-8 strings), and a second library that supports 16-bit character strings -(including UTF-16 strings). The build process allows either one or both to be -built. The majority of the work to make this possible was done by Zoltan -Herczeg. -.P -Starting with release 8.32 it is possible to compile a third separate PCRE -library that supports 32-bit character strings (including UTF-32 strings). The -build process allows any combination of the 8-, 16- and 32-bit libraries. The -work to make this possible was done by Christian Persch. -.P -The three libraries contain identical sets of functions, except that the names -in the 16-bit library start with \fBpcre16_\fP instead of \fBpcre_\fP, and the -names in the 32-bit library start with \fBpcre32_\fP instead of \fBpcre_\fP. To -avoid over-complication and reduce the documentation maintenance load, most of -the documentation describes the 8-bit library, with the differences for the -16-bit and 32-bit libraries described separately in the -.\" HREF -\fBpcre16\fP -and -.\" HREF -\fBpcre32\fP -.\" -pages. References to functions or structures of the form \fIpcre[16|32]_xxx\fP -should be read as meaning "\fIpcre_xxx\fP when using the 8-bit library, -\fIpcre16_xxx\fP when using the 16-bit library, or \fIpcre32_xxx\fP when using -the 32-bit library". -.P -The current implementation of PCRE corresponds approximately with Perl 5.12, -including support for UTF-8/16/32 encoded strings and Unicode general category -properties. However, UTF-8/16/32 and Unicode support has to be explicitly -enabled; it is not the default. The Unicode tables correspond to Unicode -release 6.3.0. -.P -In addition to the Perl-compatible matching function, PCRE contains an -alternative function that matches the same compiled patterns in a different -way. In certain circumstances, the alternative function has some advantages. -For a discussion of the two matching algorithms, see the -.\" HREF -\fBpcrematching\fP -.\" -page. -.P -PCRE is written in C and released as a C library. A number of people have -written wrappers and interfaces of various kinds. In particular, Google Inc. -have provided a comprehensive C++ wrapper for the 8-bit library. This is now -included as part of the PCRE distribution. The -.\" HREF -\fBpcrecpp\fP -.\" -page has details of this interface. Other people's contributions can be found -in the \fIContrib\fP directory at the primary FTP site, which is: -.sp -.\" HTML <a href="ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre"> -.\" </a> -ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre -.\" -.P -Details of exactly which Perl regular expression features are and are not -supported by PCRE are given in separate documents. See the -.\" HREF -\fBpcrepattern\fP -.\" -and -.\" HREF -\fBpcrecompat\fP -.\" -pages. There is a syntax summary in the -.\" HREF -\fBpcresyntax\fP -.\" -page. -.P -Some features of PCRE can be included, excluded, or changed when the library is -built. The -.\" HREF -\fBpcre_config()\fP -.\" -function makes it possible for a client to discover which features are -available. The features themselves are described in the -.\" HREF -\fBpcrebuild\fP -.\" -page. Documentation about building PCRE for various operating systems can be -found in the -.\" HTML <a href="README.txt"> -.\" </a> -\fBREADME\fP -.\" -and -.\" HTML <a href="NON-AUTOTOOLS-BUILD.txt"> -.\" </a> -\fBNON-AUTOTOOLS_BUILD\fP -.\" -files in the source distribution. -.P -The libraries contains a number of undocumented internal functions and data -tables that are used by more than one of the exported external functions, but -which are not intended for use by external callers. Their names all begin with -"_pcre_" or "_pcre16_" or "_pcre32_", which hopefully will not provoke any name -clashes. In some environments, it is possible to control which external symbols -are exported when a shared library is built, and in these cases the -undocumented symbols are not exported. -. -. -.SH "SECURITY CONSIDERATIONS" -.rs -.sp -If you are using PCRE in a non-UTF application that permits users to supply -arbitrary patterns for compilation, you should be aware of a feature that -allows users to turn on UTF support from within a pattern, provided that PCRE -was built with UTF support. For example, an 8-bit pattern that begins with -"(*UTF8)" or "(*UTF)" turns on UTF-8 mode, which interprets patterns and -subjects as strings of UTF-8 characters instead of individual 8-bit characters. -This causes both the pattern and any data against which it is matched to be -checked for UTF-8 validity. If the data string is very long, such a check might -use sufficiently many resources as to cause your application to lose -performance. -.P -One way of guarding against this possibility is to use the -\fBpcre_fullinfo()\fP function to check the compiled pattern's options for UTF. -Alternatively, from release 8.33, you can set the PCRE_NEVER_UTF option at -compile time. This causes an compile time error if a pattern contains a -UTF-setting sequence. -.P -If your application is one that supports UTF, be aware that validity checking -can take time. If the same data string is to be matched many times, you can use -the PCRE_NO_UTF[8|16|32]_CHECK option for the second and subsequent matches to -save redundant checks. -.P -Another way that performance can be hit is by running a pattern that has a very -large search tree against a string that will never match. Nested unlimited -repeats in a pattern are a common example. PCRE provides some protection -against this: see the PCRE_EXTRA_MATCH_LIMIT feature in the -.\" HREF -\fBpcreapi\fP -.\" -page. -. -. -.SH "USER DOCUMENTATION" -.rs -.sp -The user documentation for PCRE comprises a number of different sections. In -the "man" format, each of these is a separate "man page". In the HTML format, -each is a separate page, linked from the index page. In the plain text format, -the descriptions of the \fBpcregrep\fP and \fBpcretest\fP programs are in files -called \fBpcregrep.txt\fP and \fBpcretest.txt\fP, respectively. The remaining -sections, except for the \fBpcredemo\fP section (which is a program listing), -are concatenated in \fBpcre.txt\fP, for ease of searching. The sections are as -follows: -.sp - pcre this document - pcre-config show PCRE installation configuration information - pcre16 details of the 16-bit library - pcre32 details of the 32-bit library - pcreapi details of PCRE's native C API - pcrebuild building PCRE - pcrecallout details of the callout feature - pcrecompat discussion of Perl compatibility - pcrecpp details of the C++ wrapper for the 8-bit library - pcredemo a demonstration C program that uses PCRE - pcregrep description of the \fBpcregrep\fP command (8-bit only) - pcrejit discussion of the just-in-time optimization support - pcrelimits details of size and other limits - pcrematching discussion of the two matching algorithms - pcrepartial details of the partial matching facility -.\" JOIN - pcrepattern syntax and semantics of supported - regular expressions - pcreperform discussion of performance issues - pcreposix the POSIX-compatible C API for the 8-bit library - pcreprecompile details of saving and re-using precompiled patterns - pcresample discussion of the pcredemo program - pcrestack discussion of stack usage - pcresyntax quick syntax reference - pcretest description of the \fBpcretest\fP testing command - pcreunicode discussion of Unicode and UTF-8/16/32 support -.sp -In the "man" and HTML formats, there is also a short page for each C library -function, listing its arguments and results. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -.P -Putting an actual email address here seems to have been a spam magnet, so I've -taken it away. If you want to email me, use my two initials, followed by the -two digits 10, at the domain cam.ac.uk. -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 08 January 2014 -Copyright (c) 1997-2014 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcre.txt b/plugins/Pcre16/docs/doc/pcre.txt deleted file mode 100644 index 14cbb8bf2b..0000000000 --- a/plugins/Pcre16/docs/doc/pcre.txt +++ /dev/null @@ -1,10423 +0,0 @@ ------------------------------------------------------------------------------ -This file contains a concatenation of the PCRE man pages, converted to plain -text format for ease of searching with a text editor, or for use on systems -that do not have a man page processor. The small individual files that give -synopses of each function in the library have not been included. Neither has -the pcredemo program. There are separate text files for the pcregrep and -pcretest commands. ------------------------------------------------------------------------------ - - -PCRE(3) Library Functions Manual PCRE(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -INTRODUCTION - - The PCRE library is a set of functions that implement regular expres- - sion pattern matching using the same syntax and semantics as Perl, with - just a few differences. Some features that appeared in Python and PCRE - before they appeared in Perl are also available using the Python syn- - tax, there is some support for one or two .NET and Oniguruma syntax - items, and there is an option for requesting some minor changes that - give better JavaScript compatibility. - - Starting with release 8.30, it is possible to compile two separate PCRE - libraries: the original, which supports 8-bit character strings - (including UTF-8 strings), and a second library that supports 16-bit - character strings (including UTF-16 strings). The build process allows - either one or both to be built. The majority of the work to make this - possible was done by Zoltan Herczeg. - - Starting with release 8.32 it is possible to compile a third separate - PCRE library that supports 32-bit character strings (including UTF-32 - strings). The build process allows any combination of the 8-, 16- and - 32-bit libraries. The work to make this possible was done by Christian - Persch. - - The three libraries contain identical sets of functions, except that - the names in the 16-bit library start with pcre16_ instead of pcre_, - and the names in the 32-bit library start with pcre32_ instead of - pcre_. To avoid over-complication and reduce the documentation mainte- - nance load, most of the documentation describes the 8-bit library, with - the differences for the 16-bit and 32-bit libraries described sepa- - rately in the pcre16 and pcre32 pages. References to functions or - structures of the form pcre[16|32]_xxx should be read as meaning - "pcre_xxx when using the 8-bit library, pcre16_xxx when using the - 16-bit library, or pcre32_xxx when using the 32-bit library". - - The current implementation of PCRE corresponds approximately with Perl - 5.12, including support for UTF-8/16/32 encoded strings and Unicode - general category properties. However, UTF-8/16/32 and Unicode support - has to be explicitly enabled; it is not the default. The Unicode tables - correspond to Unicode release 6.3.0. - - In addition to the Perl-compatible matching function, PCRE contains an - alternative function that matches the same compiled patterns in a dif- - ferent way. In certain circumstances, the alternative function has some - advantages. For a discussion of the two matching algorithms, see the - pcrematching page. - - PCRE is written in C and released as a C library. A number of people - have written wrappers and interfaces of various kinds. In particular, - Google Inc. have provided a comprehensive C++ wrapper for the 8-bit - library. This is now included as part of the PCRE distribution. The - pcrecpp page has details of this interface. Other people's contribu- - tions can be found in the Contrib directory at the primary FTP site, - which is: - - ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre - - Details of exactly which Perl regular expression features are and are - not supported by PCRE are given in separate documents. See the pcrepat- - tern and pcrecompat pages. There is a syntax summary in the pcresyntax - page. - - Some features of PCRE can be included, excluded, or changed when the - library is built. The pcre_config() function makes it possible for a - client to discover which features are available. The features them- - selves are described in the pcrebuild page. Documentation about build- - ing PCRE for various operating systems can be found in the README and - NON-AUTOTOOLS_BUILD files in the source distribution. - - The libraries contains a number of undocumented internal functions and - data tables that are used by more than one of the exported external - functions, but which are not intended for use by external callers. - Their names all begin with "_pcre_" or "_pcre16_" or "_pcre32_", which - hopefully will not provoke any name clashes. In some environments, it - is possible to control which external symbols are exported when a - shared library is built, and in these cases the undocumented symbols - are not exported. - - -SECURITY CONSIDERATIONS - - If you are using PCRE in a non-UTF application that permits users to - supply arbitrary patterns for compilation, you should be aware of a - feature that allows users to turn on UTF support from within a pattern, - provided that PCRE was built with UTF support. For example, an 8-bit - pattern that begins with "(*UTF8)" or "(*UTF)" turns on UTF-8 mode, - which interprets patterns and subjects as strings of UTF-8 characters - instead of individual 8-bit characters. This causes both the pattern - and any data against which it is matched to be checked for UTF-8 valid- - ity. If the data string is very long, such a check might use suffi- - ciently many resources as to cause your application to lose perfor- - mance. - - One way of guarding against this possibility is to use the - pcre_fullinfo() function to check the compiled pattern's options for - UTF. Alternatively, from release 8.33, you can set the PCRE_NEVER_UTF - option at compile time. This causes an compile time error if a pattern - contains a UTF-setting sequence. - - If your application is one that supports UTF, be aware that validity - checking can take time. If the same data string is to be matched many - times, you can use the PCRE_NO_UTF[8|16|32]_CHECK option for the second - and subsequent matches to save redundant checks. - - Another way that performance can be hit is by running a pattern that - has a very large search tree against a string that will never match. - Nested unlimited repeats in a pattern are a common example. PCRE pro- - vides some protection against this: see the PCRE_EXTRA_MATCH_LIMIT fea- - ture in the pcreapi page. - - -USER DOCUMENTATION - - The user documentation for PCRE comprises a number of different sec- - tions. In the "man" format, each of these is a separate "man page". In - the HTML format, each is a separate page, linked from the index page. - In the plain text format, the descriptions of the pcregrep and pcretest - programs are in files called pcregrep.txt and pcretest.txt, respec- - tively. The remaining sections, except for the pcredemo section (which - is a program listing), are concatenated in pcre.txt, for ease of - searching. The sections are as follows: - - pcre this document - pcre-config show PCRE installation configuration information - pcre16 details of the 16-bit library - pcre32 details of the 32-bit library - pcreapi details of PCRE's native C API - pcrebuild building PCRE - pcrecallout details of the callout feature - pcrecompat discussion of Perl compatibility - pcrecpp details of the C++ wrapper for the 8-bit library - pcredemo a demonstration C program that uses PCRE - pcregrep description of the pcregrep command (8-bit only) - pcrejit discussion of the just-in-time optimization support - pcrelimits details of size and other limits - pcrematching discussion of the two matching algorithms - pcrepartial details of the partial matching facility - pcrepattern syntax and semantics of supported - regular expressions - pcreperform discussion of performance issues - pcreposix the POSIX-compatible C API for the 8-bit library - pcreprecompile details of saving and re-using precompiled patterns - pcresample discussion of the pcredemo program - pcrestack discussion of stack usage - pcresyntax quick syntax reference - pcretest description of the pcretest testing command - pcreunicode discussion of Unicode and UTF-8/16/32 support - - In the "man" and HTML formats, there is also a short page for each C - library function, listing its arguments and results. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - Putting an actual email address here seems to have been a spam magnet, - so I've taken it away. If you want to email me, use my two initials, - followed by the two digits 10, at the domain cam.ac.uk. - - -REVISION - - Last updated: 08 January 2014 - Copyright (c) 1997-2014 University of Cambridge. ------------------------------------------------------------------------------- - - -PCRE(3) Library Functions Manual PCRE(3) - - - -NAME - PCRE - Perl-compatible regular expressions - - #include <pcre.h> - - -PCRE 16-BIT API BASIC FUNCTIONS - - pcre16 *pcre16_compile(PCRE_SPTR16 pattern, int options, - const char **errptr, int *erroffset, - const unsigned char *tableptr); - - pcre16 *pcre16_compile2(PCRE_SPTR16 pattern, int options, - int *errorcodeptr, - const char **errptr, int *erroffset, - const unsigned char *tableptr); - - pcre16_extra *pcre16_study(const pcre16 *code, int options, - const char **errptr); - - void pcre16_free_study(pcre16_extra *extra); - - int pcre16_exec(const pcre16 *code, const pcre16_extra *extra, - PCRE_SPTR16 subject, int length, int startoffset, - int options, int *ovector, int ovecsize); - - int pcre16_dfa_exec(const pcre16 *code, const pcre16_extra *extra, - PCRE_SPTR16 subject, int length, int startoffset, - int options, int *ovector, int ovecsize, - int *workspace, int wscount); - - -PCRE 16-BIT API STRING EXTRACTION FUNCTIONS - - int pcre16_copy_named_substring(const pcre16 *code, - PCRE_SPTR16 subject, int *ovector, - int stringcount, PCRE_SPTR16 stringname, - PCRE_UCHAR16 *buffer, int buffersize); - - int pcre16_copy_substring(PCRE_SPTR16 subject, int *ovector, - int stringcount, int stringnumber, PCRE_UCHAR16 *buffer, - int buffersize); - - int pcre16_get_named_substring(const pcre16 *code, - PCRE_SPTR16 subject, int *ovector, - int stringcount, PCRE_SPTR16 stringname, - PCRE_SPTR16 *stringptr); - - int pcre16_get_stringnumber(const pcre16 *code, - PCRE_SPTR16 name); - - int pcre16_get_stringtable_entries(const pcre16 *code, - PCRE_SPTR16 name, PCRE_UCHAR16 **first, PCRE_UCHAR16 **last); - - int pcre16_get_substring(PCRE_SPTR16 subject, int *ovector, - int stringcount, int stringnumber, - PCRE_SPTR16 *stringptr); - - int pcre16_get_substring_list(PCRE_SPTR16 subject, - int *ovector, int stringcount, PCRE_SPTR16 **listptr); - - void pcre16_free_substring(PCRE_SPTR16 stringptr); - - void pcre16_free_substring_list(PCRE_SPTR16 *stringptr); - - -PCRE 16-BIT API AUXILIARY FUNCTIONS - - pcre16_jit_stack *pcre16_jit_stack_alloc(int startsize, int maxsize); - - void pcre16_jit_stack_free(pcre16_jit_stack *stack); - - void pcre16_assign_jit_stack(pcre16_extra *extra, - pcre16_jit_callback callback, void *data); - - const unsigned char *pcre16_maketables(void); - - int pcre16_fullinfo(const pcre16 *code, const pcre16_extra *extra, - int what, void *where); - - int pcre16_refcount(pcre16 *code, int adjust); - - int pcre16_config(int what, void *where); - - const char *pcre16_version(void); - - int pcre16_pattern_to_host_byte_order(pcre16 *code, - pcre16_extra *extra, const unsigned char *tables); - - -PCRE 16-BIT API INDIRECTED FUNCTIONS - - void *(*pcre16_malloc)(size_t); - - void (*pcre16_free)(void *); - - void *(*pcre16_stack_malloc)(size_t); - - void (*pcre16_stack_free)(void *); - - int (*pcre16_callout)(pcre16_callout_block *); - - -PCRE 16-BIT API 16-BIT-ONLY FUNCTION - - int pcre16_utf16_to_host_byte_order(PCRE_UCHAR16 *output, - PCRE_SPTR16 input, int length, int *byte_order, - int keep_boms); - - -THE PCRE 16-BIT LIBRARY - - Starting with release 8.30, it is possible to compile a PCRE library - that supports 16-bit character strings, including UTF-16 strings, as - well as or instead of the original 8-bit library. The majority of the - work to make this possible was done by Zoltan Herczeg. The two - libraries contain identical sets of functions, used in exactly the same - way. Only the names of the functions and the data types of their argu- - ments and results are different. To avoid over-complication and reduce - the documentation maintenance load, most of the PCRE documentation - describes the 8-bit library, with only occasional references to the - 16-bit library. This page describes what is different when you use the - 16-bit library. - - WARNING: A single application can be linked with both libraries, but - you must take care when processing any particular pattern to use func- - tions from just one library. For example, if you want to study a pat- - tern that was compiled with pcre16_compile(), you must do so with - pcre16_study(), not pcre_study(), and you must free the study data with - pcre16_free_study(). - - -THE HEADER FILE - - There is only one header file, pcre.h. It contains prototypes for all - the functions in all libraries, as well as definitions of flags, struc- - tures, error codes, etc. - - -THE LIBRARY NAME - - In Unix-like systems, the 16-bit library is called libpcre16, and can - normally be accesss by adding -lpcre16 to the command for linking an - application that uses PCRE. - - -STRING TYPES - - In the 8-bit library, strings are passed to PCRE library functions as - vectors of bytes with the C type "char *". In the 16-bit library, - strings are passed as vectors of unsigned 16-bit quantities. The macro - PCRE_UCHAR16 specifies an appropriate data type, and PCRE_SPTR16 is - defined as "const PCRE_UCHAR16 *". In very many environments, "short - int" is a 16-bit data type. When PCRE is built, it defines PCRE_UCHAR16 - as "unsigned short int", but checks that it really is a 16-bit data - type. If it is not, the build fails with an error message telling the - maintainer to modify the definition appropriately. - - -STRUCTURE TYPES - - The types of the opaque structures that are used for compiled 16-bit - patterns and JIT stacks are pcre16 and pcre16_jit_stack respectively. - The type of the user-accessible structure that is returned by - pcre16_study() is pcre16_extra, and the type of the structure that is - used for passing data to a callout function is pcre16_callout_block. - These structures contain the same fields, with the same names, as their - 8-bit counterparts. The only difference is that pointers to character - strings are 16-bit instead of 8-bit types. - - -16-BIT FUNCTIONS - - For every function in the 8-bit library there is a corresponding func- - tion in the 16-bit library with a name that starts with pcre16_ instead - of pcre_. The prototypes are listed above. In addition, there is one - extra function, pcre16_utf16_to_host_byte_order(). This is a utility - function that converts a UTF-16 character string to host byte order if - necessary. The other 16-bit functions expect the strings they are - passed to be in host byte order. - - The input and output arguments of pcre16_utf16_to_host_byte_order() may - point to the same address, that is, conversion in place is supported. - The output buffer must be at least as long as the input. - - The length argument specifies the number of 16-bit data units in the - input string; a negative value specifies a zero-terminated string. - - If byte_order is NULL, it is assumed that the string starts off in host - byte order. This may be changed by byte-order marks (BOMs) anywhere in - the string (commonly as the first character). - - If byte_order is not NULL, a non-zero value of the integer to which it - points means that the input starts off in host byte order, otherwise - the opposite order is assumed. Again, BOMs in the string can change - this. The final byte order is passed back at the end of processing. - - If keep_boms is not zero, byte-order mark characters (0xfeff) are - copied into the output string. Otherwise they are discarded. - - The result of the function is the number of 16-bit units placed into - the output buffer, including the zero terminator if the string was - zero-terminated. - - -SUBJECT STRING OFFSETS - - The lengths and starting offsets of subject strings must be specified - in 16-bit data units, and the offsets within subject strings that are - returned by the matching functions are in also 16-bit units rather than - bytes. - - -NAMED SUBPATTERNS - - The name-to-number translation table that is maintained for named sub- - patterns uses 16-bit characters. The pcre16_get_stringtable_entries() - function returns the length of each entry in the table as the number of - 16-bit data units. - - -OPTION NAMES - - There are two new general option names, PCRE_UTF16 and - PCRE_NO_UTF16_CHECK, which correspond to PCRE_UTF8 and - PCRE_NO_UTF8_CHECK in the 8-bit library. In fact, these new options - define the same bits in the options word. There is a discussion about - the validity of UTF-16 strings in the pcreunicode page. - - For the pcre16_config() function there is an option PCRE_CONFIG_UTF16 - that returns 1 if UTF-16 support is configured, otherwise 0. If this - option is given to pcre_config() or pcre32_config(), or if the - PCRE_CONFIG_UTF8 or PCRE_CONFIG_UTF32 option is given to pcre16_con- - fig(), the result is the PCRE_ERROR_BADOPTION error. - - -CHARACTER CODES - - In 16-bit mode, when PCRE_UTF16 is not set, character values are - treated in the same way as in 8-bit, non UTF-8 mode, except, of course, - that they can range from 0 to 0xffff instead of 0 to 0xff. Character - types for characters less than 0xff can therefore be influenced by the - locale in the same way as before. Characters greater than 0xff have - only one case, and no "type" (such as letter or digit). - - In UTF-16 mode, the character code is Unicode, in the range 0 to - 0x10ffff, with the exception of values in the range 0xd800 to 0xdfff - because those are "surrogate" values that are used in pairs to encode - values greater than 0xffff. - - A UTF-16 string can indicate its endianness by special code knows as a - byte-order mark (BOM). The PCRE functions do not handle this, expecting - strings to be in host byte order. A utility function called - pcre16_utf16_to_host_byte_order() is provided to help with this (see - above). - - -ERROR NAMES - - The errors PCRE_ERROR_BADUTF16_OFFSET and PCRE_ERROR_SHORTUTF16 corre- - spond to their 8-bit counterparts. The error PCRE_ERROR_BADMODE is - given when a compiled pattern is passed to a function that processes - patterns in the other mode, for example, if a pattern compiled with - pcre_compile() is passed to pcre16_exec(). - - There are new error codes whose names begin with PCRE_UTF16_ERR for - invalid UTF-16 strings, corresponding to the PCRE_UTF8_ERR codes for - UTF-8 strings that are described in the section entitled "Reason codes - for invalid UTF-8 strings" in the main pcreapi page. The UTF-16 errors - are: - - PCRE_UTF16_ERR1 Missing low surrogate at end of string - PCRE_UTF16_ERR2 Invalid low surrogate follows high surrogate - PCRE_UTF16_ERR3 Isolated low surrogate - PCRE_UTF16_ERR4 Non-character - - -ERROR TEXTS - - If there is an error while compiling a pattern, the error text that is - passed back by pcre16_compile() or pcre16_compile2() is still an 8-bit - character string, zero-terminated. - - -CALLOUTS - - The subject and mark fields in the callout block that is passed to a - callout function point to 16-bit vectors. - - -TESTING - - The pcretest program continues to operate with 8-bit input and output - files, but it can be used for testing the 16-bit library. If it is run - with the command line option -16, patterns and subject strings are con- - verted from 8-bit to 16-bit before being passed to PCRE, and the 16-bit - library functions are used instead of the 8-bit ones. Returned 16-bit - strings are converted to 8-bit for output. If both the 8-bit and the - 32-bit libraries were not compiled, pcretest defaults to 16-bit and the - -16 option is ignored. - - When PCRE is being built, the RunTest script that is called by "make - check" uses the pcretest -C option to discover which of the 8-bit, - 16-bit and 32-bit libraries has been built, and runs the tests appro- - priately. - - -NOT SUPPORTED IN 16-BIT MODE - - Not all the features of the 8-bit library are available with the 16-bit - library. The C++ and POSIX wrapper functions support only the 8-bit - library, and the pcregrep program is at present 8-bit only. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 12 May 2013 - Copyright (c) 1997-2013 University of Cambridge. ------------------------------------------------------------------------------- - - -PCRE(3) Library Functions Manual PCRE(3) - - - -NAME - PCRE - Perl-compatible regular expressions - - #include <pcre.h> - - -PCRE 32-BIT API BASIC FUNCTIONS - - pcre32 *pcre32_compile(PCRE_SPTR32 pattern, int options, - const char **errptr, int *erroffset, - const unsigned char *tableptr); - - pcre32 *pcre32_compile2(PCRE_SPTR32 pattern, int options, - int *errorcodeptr, - const unsigned char *tableptr); - - pcre32_extra *pcre32_study(const pcre32 *code, int options, - const char **errptr); - - void pcre32_free_study(pcre32_extra *extra); - - int pcre32_exec(const pcre32 *code, const pcre32_extra *extra, - PCRE_SPTR32 subject, int length, int startoffset, - int options, int *ovector, int ovecsize); - - int pcre32_dfa_exec(const pcre32 *code, const pcre32_extra *extra, - PCRE_SPTR32 subject, int length, int startoffset, - int options, int *ovector, int ovecsize, - int *workspace, int wscount); - - -PCRE 32-BIT API STRING EXTRACTION FUNCTIONS - - int pcre32_copy_named_substring(const pcre32 *code, - PCRE_SPTR32 subject, int *ovector, - int stringcount, PCRE_SPTR32 stringname, - PCRE_UCHAR32 *buffer, int buffersize); - - int pcre32_copy_substring(PCRE_SPTR32 subject, int *ovector, - int stringcount, int stringnumber, PCRE_UCHAR32 *buffer, - int buffersize); - - int pcre32_get_named_substring(const pcre32 *code, - PCRE_SPTR32 subject, int *ovector, - int stringcount, PCRE_SPTR32 stringname, - PCRE_SPTR32 *stringptr); - - int pcre32_get_stringnumber(const pcre32 *code, - PCRE_SPTR32 name); - - int pcre32_get_stringtable_entries(const pcre32 *code, - PCRE_SPTR32 name, PCRE_UCHAR32 **first, PCRE_UCHAR32 **last); - - int pcre32_get_substring(PCRE_SPTR32 subject, int *ovector, - int stringcount, int stringnumber, - PCRE_SPTR32 *stringptr); - - int pcre32_get_substring_list(PCRE_SPTR32 subject, - int *ovector, int stringcount, PCRE_SPTR32 **listptr); - - void pcre32_free_substring(PCRE_SPTR32 stringptr); - - void pcre32_free_substring_list(PCRE_SPTR32 *stringptr); - - -PCRE 32-BIT API AUXILIARY FUNCTIONS - - pcre32_jit_stack *pcre32_jit_stack_alloc(int startsize, int maxsize); - - void pcre32_jit_stack_free(pcre32_jit_stack *stack); - - void pcre32_assign_jit_stack(pcre32_extra *extra, - pcre32_jit_callback callback, void *data); - - const unsigned char *pcre32_maketables(void); - - int pcre32_fullinfo(const pcre32 *code, const pcre32_extra *extra, - int what, void *where); - - int pcre32_refcount(pcre32 *code, int adjust); - - int pcre32_config(int what, void *where); - - const char *pcre32_version(void); - - int pcre32_pattern_to_host_byte_order(pcre32 *code, - pcre32_extra *extra, const unsigned char *tables); - - -PCRE 32-BIT API INDIRECTED FUNCTIONS - - void *(*pcre32_malloc)(size_t); - - void (*pcre32_free)(void *); - - void *(*pcre32_stack_malloc)(size_t); - - void (*pcre32_stack_free)(void *); - - int (*pcre32_callout)(pcre32_callout_block *); - - -PCRE 32-BIT API 32-BIT-ONLY FUNCTION - - int pcre32_utf32_to_host_byte_order(PCRE_UCHAR32 *output, - PCRE_SPTR32 input, int length, int *byte_order, - int keep_boms); - - -THE PCRE 32-BIT LIBRARY - - Starting with release 8.32, it is possible to compile a PCRE library - that supports 32-bit character strings, including UTF-32 strings, as - well as or instead of the original 8-bit library. This work was done by - Christian Persch, based on the work done by Zoltan Herczeg for the - 16-bit library. All three libraries contain identical sets of func- - tions, used in exactly the same way. Only the names of the functions - and the data types of their arguments and results are different. To - avoid over-complication and reduce the documentation maintenance load, - most of the PCRE documentation describes the 8-bit library, with only - occasional references to the 16-bit and 32-bit libraries. This page - describes what is different when you use the 32-bit library. - - WARNING: A single application can be linked with all or any of the - three libraries, but you must take care when processing any particular - pattern to use functions from just one library. For example, if you - want to study a pattern that was compiled with pcre32_compile(), you - must do so with pcre32_study(), not pcre_study(), and you must free the - study data with pcre32_free_study(). - - -THE HEADER FILE - - There is only one header file, pcre.h. It contains prototypes for all - the functions in all libraries, as well as definitions of flags, struc- - tures, error codes, etc. - - -THE LIBRARY NAME - - In Unix-like systems, the 32-bit library is called libpcre32, and can - normally be accesss by adding -lpcre32 to the command for linking an - application that uses PCRE. - - -STRING TYPES - - In the 8-bit library, strings are passed to PCRE library functions as - vectors of bytes with the C type "char *". In the 32-bit library, - strings are passed as vectors of unsigned 32-bit quantities. The macro - PCRE_UCHAR32 specifies an appropriate data type, and PCRE_SPTR32 is - defined as "const PCRE_UCHAR32 *". In very many environments, "unsigned - int" is a 32-bit data type. When PCRE is built, it defines PCRE_UCHAR32 - as "unsigned int", but checks that it really is a 32-bit data type. If - it is not, the build fails with an error message telling the maintainer - to modify the definition appropriately. - - -STRUCTURE TYPES - - The types of the opaque structures that are used for compiled 32-bit - patterns and JIT stacks are pcre32 and pcre32_jit_stack respectively. - The type of the user-accessible structure that is returned by - pcre32_study() is pcre32_extra, and the type of the structure that is - used for passing data to a callout function is pcre32_callout_block. - These structures contain the same fields, with the same names, as their - 8-bit counterparts. The only difference is that pointers to character - strings are 32-bit instead of 8-bit types. - - -32-BIT FUNCTIONS - - For every function in the 8-bit library there is a corresponding func- - tion in the 32-bit library with a name that starts with pcre32_ instead - of pcre_. The prototypes are listed above. In addition, there is one - extra function, pcre32_utf32_to_host_byte_order(). This is a utility - function that converts a UTF-32 character string to host byte order if - necessary. The other 32-bit functions expect the strings they are - passed to be in host byte order. - - The input and output arguments of pcre32_utf32_to_host_byte_order() may - point to the same address, that is, conversion in place is supported. - The output buffer must be at least as long as the input. - - The length argument specifies the number of 32-bit data units in the - input string; a negative value specifies a zero-terminated string. - - If byte_order is NULL, it is assumed that the string starts off in host - byte order. This may be changed by byte-order marks (BOMs) anywhere in - the string (commonly as the first character). - - If byte_order is not NULL, a non-zero value of the integer to which it - points means that the input starts off in host byte order, otherwise - the opposite order is assumed. Again, BOMs in the string can change - this. The final byte order is passed back at the end of processing. - - If keep_boms is not zero, byte-order mark characters (0xfeff) are - copied into the output string. Otherwise they are discarded. - - The result of the function is the number of 32-bit units placed into - the output buffer, including the zero terminator if the string was - zero-terminated. - - -SUBJECT STRING OFFSETS - - The lengths and starting offsets of subject strings must be specified - in 32-bit data units, and the offsets within subject strings that are - returned by the matching functions are in also 32-bit units rather than - bytes. - - -NAMED SUBPATTERNS - - The name-to-number translation table that is maintained for named sub- - patterns uses 32-bit characters. The pcre32_get_stringtable_entries() - function returns the length of each entry in the table as the number of - 32-bit data units. - - -OPTION NAMES - - There are two new general option names, PCRE_UTF32 and - PCRE_NO_UTF32_CHECK, which correspond to PCRE_UTF8 and - PCRE_NO_UTF8_CHECK in the 8-bit library. In fact, these new options - define the same bits in the options word. There is a discussion about - the validity of UTF-32 strings in the pcreunicode page. - - For the pcre32_config() function there is an option PCRE_CONFIG_UTF32 - that returns 1 if UTF-32 support is configured, otherwise 0. If this - option is given to pcre_config() or pcre16_config(), or if the - PCRE_CONFIG_UTF8 or PCRE_CONFIG_UTF16 option is given to pcre32_con- - fig(), the result is the PCRE_ERROR_BADOPTION error. - - -CHARACTER CODES - - In 32-bit mode, when PCRE_UTF32 is not set, character values are - treated in the same way as in 8-bit, non UTF-8 mode, except, of course, - that they can range from 0 to 0x7fffffff instead of 0 to 0xff. Charac- - ter types for characters less than 0xff can therefore be influenced by - the locale in the same way as before. Characters greater than 0xff - have only one case, and no "type" (such as letter or digit). - - In UTF-32 mode, the character code is Unicode, in the range 0 to - 0x10ffff, with the exception of values in the range 0xd800 to 0xdfff - because those are "surrogate" values that are ill-formed in UTF-32. - - A UTF-32 string can indicate its endianness by special code knows as a - byte-order mark (BOM). The PCRE functions do not handle this, expecting - strings to be in host byte order. A utility function called - pcre32_utf32_to_host_byte_order() is provided to help with this (see - above). - - -ERROR NAMES - - The error PCRE_ERROR_BADUTF32 corresponds to its 8-bit counterpart. - The error PCRE_ERROR_BADMODE is given when a compiled pattern is passed - to a function that processes patterns in the other mode, for example, - if a pattern compiled with pcre_compile() is passed to pcre32_exec(). - - There are new error codes whose names begin with PCRE_UTF32_ERR for - invalid UTF-32 strings, corresponding to the PCRE_UTF8_ERR codes for - UTF-8 strings that are described in the section entitled "Reason codes - for invalid UTF-8 strings" in the main pcreapi page. The UTF-32 errors - are: - - PCRE_UTF32_ERR1 Surrogate character (range from 0xd800 to 0xdfff) - PCRE_UTF32_ERR2 Non-character - PCRE_UTF32_ERR3 Character > 0x10ffff - - -ERROR TEXTS - - If there is an error while compiling a pattern, the error text that is - passed back by pcre32_compile() or pcre32_compile2() is still an 8-bit - character string, zero-terminated. - - -CALLOUTS - - The subject and mark fields in the callout block that is passed to a - callout function point to 32-bit vectors. - - -TESTING - - The pcretest program continues to operate with 8-bit input and output - files, but it can be used for testing the 32-bit library. If it is run - with the command line option -32, patterns and subject strings are con- - verted from 8-bit to 32-bit before being passed to PCRE, and the 32-bit - library functions are used instead of the 8-bit ones. Returned 32-bit - strings are converted to 8-bit for output. If both the 8-bit and the - 16-bit libraries were not compiled, pcretest defaults to 32-bit and the - -32 option is ignored. - - When PCRE is being built, the RunTest script that is called by "make - check" uses the pcretest -C option to discover which of the 8-bit, - 16-bit and 32-bit libraries has been built, and runs the tests appro- - priately. - - -NOT SUPPORTED IN 32-BIT MODE - - Not all the features of the 8-bit library are available with the 32-bit - library. The C++ and POSIX wrapper functions support only the 8-bit - library, and the pcregrep program is at present 8-bit only. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 12 May 2013 - Copyright (c) 1997-2013 University of Cambridge. ------------------------------------------------------------------------------- - - -PCREBUILD(3) Library Functions Manual PCREBUILD(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -BUILDING PCRE - - PCRE is distributed with a configure script that can be used to build - the library in Unix-like environments using the applications known as - Autotools. Also in the distribution are files to support building - using CMake instead of configure. The text file README contains general - information about building with Autotools (some of which is repeated - below), and also has some comments about building on various operating - systems. There is a lot more information about building PCRE without - using Autotools (including information about using CMake and building - "by hand") in the text file called NON-AUTOTOOLS-BUILD. You should - consult this file as well as the README file if you are building in a - non-Unix-like environment. - - -PCRE BUILD-TIME OPTIONS - - The rest of this document describes the optional features of PCRE that - can be selected when the library is compiled. It assumes use of the - configure script, where the optional features are selected or dese- - lected by providing options to configure before running the make com- - mand. However, the same options can be selected in both Unix-like and - non-Unix-like environments using the GUI facility of cmake-gui if you - are using CMake instead of configure to build PCRE. - - If you are not using Autotools or CMake, option selection can be done - by editing the config.h file, or by passing parameter settings to the - compiler, as described in NON-AUTOTOOLS-BUILD. - - The complete list of options for configure (which includes the standard - ones such as the selection of the installation directory) can be - obtained by running - - ./configure --help - - The following sections include descriptions of options whose names - begin with --enable or --disable. These settings specify changes to the - defaults for the configure command. Because of the way that configure - works, --enable and --disable always come in pairs, so the complemen- - tary option always exists as well, but as it specifies the default, it - is not described. - - -BUILDING 8-BIT, 16-BIT AND 32-BIT LIBRARIES - - By default, a library called libpcre is built, containing functions - that take string arguments contained in vectors of bytes, either as - single-byte characters, or interpreted as UTF-8 strings. You can also - build a separate library, called libpcre16, in which strings are con- - tained in vectors of 16-bit data units and interpreted either as sin- - gle-unit characters or UTF-16 strings, by adding - - --enable-pcre16 - - to the configure command. You can also build yet another separate - library, called libpcre32, in which strings are contained in vectors of - 32-bit data units and interpreted either as single-unit characters or - UTF-32 strings, by adding - - --enable-pcre32 - - to the configure command. If you do not want the 8-bit library, add - - --disable-pcre8 - - as well. At least one of the three libraries must be built. Note that - the C++ and POSIX wrappers are for the 8-bit library only, and that - pcregrep is an 8-bit program. None of these are built if you select - only the 16-bit or 32-bit libraries. - - -BUILDING SHARED AND STATIC LIBRARIES - - The Autotools PCRE building process uses libtool to build both shared - and static libraries by default. You can suppress one of these by - adding one of - - --disable-shared - --disable-static - - to the configure command, as required. - - -C++ SUPPORT - - By default, if the 8-bit library is being built, the configure script - will search for a C++ compiler and C++ header files. If it finds them, - it automatically builds the C++ wrapper library (which supports only - 8-bit strings). You can disable this by adding - - --disable-cpp - - to the configure command. - - -UTF-8, UTF-16 AND UTF-32 SUPPORT - - To build PCRE with support for UTF Unicode character strings, add - - --enable-utf - - to the configure command. This setting applies to all three libraries, - adding support for UTF-8 to the 8-bit library, support for UTF-16 to - the 16-bit library, and support for UTF-32 to the to the 32-bit - library. There are no separate options for enabling UTF-8, UTF-16 and - UTF-32 independently because that would allow ridiculous settings such - as requesting UTF-16 support while building only the 8-bit library. It - is not possible to build one library with UTF support and another with- - out in the same configuration. (For backwards compatibility, --enable- - utf8 is a synonym of --enable-utf.) - - Of itself, this setting does not make PCRE treat strings as UTF-8, - UTF-16 or UTF-32. As well as compiling PCRE with this option, you also - have have to set the PCRE_UTF8, PCRE_UTF16 or PCRE_UTF32 option (as - appropriate) when you call one of the pattern compiling functions. - - If you set --enable-utf when compiling in an EBCDIC environment, PCRE - expects its input to be either ASCII or UTF-8 (depending on the run- - time option). It is not possible to support both EBCDIC and UTF-8 codes - in the same version of the library. Consequently, --enable-utf and - --enable-ebcdic are mutually exclusive. - - -UNICODE CHARACTER PROPERTY SUPPORT - - UTF support allows the libraries to process character codepoints up to - 0x10ffff in the strings that they handle. On its own, however, it does - not provide any facilities for accessing the properties of such charac- - ters. If you want to be able to use the pattern escapes \P, \p, and \X, - which refer to Unicode character properties, you must add - - --enable-unicode-properties - - to the configure command. This implies UTF support, even if you have - not explicitly requested it. - - Including Unicode property support adds around 30K of tables to the - PCRE library. Only the general category properties such as Lu and Nd - are supported. Details are given in the pcrepattern documentation. - - -JUST-IN-TIME COMPILER SUPPORT - - Just-in-time compiler support is included in the build by specifying - - --enable-jit - - This support is available only for certain hardware architectures. If - this option is set for an unsupported architecture, a compile time - error occurs. See the pcrejit documentation for a discussion of JIT - usage. When JIT support is enabled, pcregrep automatically makes use of - it, unless you add - - --disable-pcregrep-jit - - to the "configure" command. - - -CODE VALUE OF NEWLINE - - By default, PCRE interprets the linefeed (LF) character as indicating - the end of a line. This is the normal newline character on Unix-like - systems. You can compile PCRE to use carriage return (CR) instead, by - adding - - --enable-newline-is-cr - - to the configure command. There is also a --enable-newline-is-lf - option, which explicitly specifies linefeed as the newline character. - - Alternatively, you can specify that line endings are to be indicated by - the two character sequence CRLF. If you want this, add - - --enable-newline-is-crlf - - to the configure command. There is a fourth option, specified by - - --enable-newline-is-anycrlf - - which causes PCRE to recognize any of the three sequences CR, LF, or - CRLF as indicating a line ending. Finally, a fifth option, specified by - - --enable-newline-is-any - - causes PCRE to recognize any Unicode newline sequence. - - Whatever line ending convention is selected when PCRE is built can be - overridden when the library functions are called. At build time it is - conventional to use the standard for your operating system. - - -WHAT \R MATCHES - - By default, the sequence \R in a pattern matches any Unicode newline - sequence, whatever has been selected as the line ending sequence. If - you specify - - --enable-bsr-anycrlf - - the default is changed so that \R matches only CR, LF, or CRLF. What- - ever is selected when PCRE is built can be overridden when the library - functions are called. - - -POSIX MALLOC USAGE - - When the 8-bit library is called through the POSIX interface (see the - pcreposix documentation), additional working storage is required for - holding the pointers to capturing substrings, because PCRE requires - three integers per substring, whereas the POSIX interface provides only - two. If the number of expected substrings is small, the wrapper func- - tion uses space on the stack, because this is faster than using mal- - loc() for each call. The default threshold above which the stack is no - longer used is 10; it can be changed by adding a setting such as - - --with-posix-malloc-threshold=20 - - to the configure command. - - -HANDLING VERY LARGE PATTERNS - - Within a compiled pattern, offset values are used to point from one - part to another (for example, from an opening parenthesis to an alter- - nation metacharacter). By default, in the 8-bit and 16-bit libraries, - two-byte values are used for these offsets, leading to a maximum size - for a compiled pattern of around 64K. This is sufficient to handle all - but the most gigantic patterns. Nevertheless, some people do want to - process truly enormous patterns, so it is possible to compile PCRE to - use three-byte or four-byte offsets by adding a setting such as - - --with-link-size=3 - - to the configure command. The value given must be 2, 3, or 4. For the - 16-bit library, a value of 3 is rounded up to 4. In these libraries, - using longer offsets slows down the operation of PCRE because it has to - load additional data when handling them. For the 32-bit library the - value is always 4 and cannot be overridden; the value of --with-link- - size is ignored. - - -AVOIDING EXCESSIVE STACK USAGE - - When matching with the pcre_exec() function, PCRE implements backtrack- - ing by making recursive calls to an internal function called match(). - In environments where the size of the stack is limited, this can se- - verely limit PCRE's operation. (The Unix environment does not usually - suffer from this problem, but it may sometimes be necessary to increase - the maximum stack size. There is a discussion in the pcrestack docu- - mentation.) An alternative approach to recursion that uses memory from - the heap to remember data, instead of using recursive function calls, - has been implemented to work round the problem of limited stack size. - If you want to build a version of PCRE that works this way, add - - --disable-stack-for-recursion - - to the configure command. With this configuration, PCRE will use the - pcre_stack_malloc and pcre_stack_free variables to call memory manage- - ment functions. By default these point to malloc() and free(), but you - can replace the pointers so that your own functions are used instead. - - Separate functions are provided rather than using pcre_malloc and - pcre_free because the usage is very predictable: the block sizes - requested are always the same, and the blocks are always freed in - reverse order. A calling program might be able to implement optimized - functions that perform better than malloc() and free(). PCRE runs - noticeably more slowly when built in this way. This option affects only - the pcre_exec() function; it is not relevant for pcre_dfa_exec(). - - -LIMITING PCRE RESOURCE USAGE - - Internally, PCRE has a function called match(), which it calls repeat- - edly (sometimes recursively) when matching a pattern with the - pcre_exec() function. By controlling the maximum number of times this - function may be called during a single matching operation, a limit can - be placed on the resources used by a single call to pcre_exec(). The - limit can be changed at run time, as described in the pcreapi documen- - tation. The default is 10 million, but this can be changed by adding a - setting such as - - --with-match-limit=500000 - - to the configure command. This setting has no effect on the - pcre_dfa_exec() matching function. - - In some environments it is desirable to limit the depth of recursive - calls of match() more strictly than the total number of calls, in order - to restrict the maximum amount of stack (or heap, if --disable-stack- - for-recursion is specified) that is used. A second limit controls this; - it defaults to the value that is set for --with-match-limit, which - imposes no additional constraints. However, you can set a lower limit - by adding, for example, - - --with-match-limit-recursion=10000 - - to the configure command. This value can also be overridden at run - time. - - -CREATING CHARACTER TABLES AT BUILD TIME - - PCRE uses fixed tables for processing characters whose code values are - less than 256. By default, PCRE is built with a set of tables that are - distributed in the file pcre_chartables.c.dist. These tables are for - ASCII codes only. If you add - - --enable-rebuild-chartables - - to the configure command, the distributed tables are no longer used. - Instead, a program called dftables is compiled and run. This outputs - the source for new set of tables, created in the default locale of your - C run-time system. (This method of replacing the tables does not work - if you are cross compiling, because dftables is run on the local host. - If you need to create alternative tables when cross compiling, you will - have to do so "by hand".) - - -USING EBCDIC CODE - - PCRE assumes by default that it will run in an environment where the - character code is ASCII (or Unicode, which is a superset of ASCII). - This is the case for most computer operating systems. PCRE can, how- - ever, be compiled to run in an EBCDIC environment by adding - - --enable-ebcdic - - to the configure command. This setting implies --enable-rebuild-charta- - bles. You should only use it if you know that you are in an EBCDIC - environment (for example, an IBM mainframe operating system). The - --enable-ebcdic option is incompatible with --enable-utf. - - The EBCDIC character that corresponds to an ASCII LF is assumed to have - the value 0x15 by default. However, in some EBCDIC environments, 0x25 - is used. In such an environment you should use - - --enable-ebcdic-nl25 - - as well as, or instead of, --enable-ebcdic. The EBCDIC character for CR - has the same value as in ASCII, namely, 0x0d. Whichever of 0x15 and - 0x25 is not chosen as LF is made to correspond to the Unicode NEL char- - acter (which, in Unicode, is 0x85). - - The options that select newline behaviour, such as --enable-newline-is- - cr, and equivalent run-time options, refer to these character values in - an EBCDIC environment. - - -PCREGREP OPTIONS FOR COMPRESSED FILE SUPPORT - - By default, pcregrep reads all files as plain text. You can build it so - that it recognizes files whose names end in .gz or .bz2, and reads them - with libz or libbz2, respectively, by adding one or both of - - --enable-pcregrep-libz - --enable-pcregrep-libbz2 - - to the configure command. These options naturally require that the rel- - evant libraries are installed on your system. Configuration will fail - if they are not. - - -PCREGREP BUFFER SIZE - - pcregrep uses an internal buffer to hold a "window" on the file it is - scanning, in order to be able to output "before" and "after" lines when - it finds a match. The size of the buffer is controlled by a parameter - whose default value is 20K. The buffer itself is three times this size, - but because of the way it is used for holding "before" lines, the long- - est line that is guaranteed to be processable is the parameter size. - You can change the default parameter value by adding, for example, - - --with-pcregrep-bufsize=50K - - to the configure command. The caller of pcregrep can, however, override - this value by specifying a run-time option. - - -PCRETEST OPTION FOR LIBREADLINE SUPPORT - - If you add - - --enable-pcretest-libreadline - - to the configure command, pcretest is linked with the libreadline - library, and when its input is from a terminal, it reads it using the - readline() function. This provides line-editing and history facilities. - Note that libreadline is GPL-licensed, so if you distribute a binary of - pcretest linked in this way, there may be licensing issues. - - Setting this option causes the -lreadline option to be added to the - pcretest build. In many operating environments with a sytem-installed - libreadline this is sufficient. However, in some environments (e.g. if - an unmodified distribution version of readline is in use), some extra - configuration may be necessary. The INSTALL file for libreadline says - this: - - "Readline uses the termcap functions, but does not link with the - termcap or curses library itself, allowing applications which link - with readline the to choose an appropriate library." - - If your environment has not been set up so that an appropriate library - is automatically included, you may need to add something like - - LIBS="-ncurses" - - immediately before the configure command. - - -DEBUGGING WITH VALGRIND SUPPORT - - By adding the - - --enable-valgrind - - option to to the configure command, PCRE will use valgrind annotations - to mark certain memory regions as unaddressable. This allows it to - detect invalid memory accesses, and is mostly useful for debugging PCRE - itself. - - -CODE COVERAGE REPORTING - - If your C compiler is gcc, you can build a version of PCRE that can - generate a code coverage report for its test suite. To enable this, you - must install lcov version 1.6 or above. Then specify - - --enable-coverage - - to the configure command and build PCRE in the usual way. - - Note that using ccache (a caching C compiler) is incompatible with code - coverage reporting. If you have configured ccache to run automatically - on your system, you must set the environment variable - - CCACHE_DISABLE=1 - - before running make to build PCRE, so that ccache is not used. - - When --enable-coverage is used, the following addition targets are - added to the Makefile: - - make coverage - - This creates a fresh coverage report for the PCRE test suite. It is - equivalent to running "make coverage-reset", "make coverage-baseline", - "make check", and then "make coverage-report". - - make coverage-reset - - This zeroes the coverage counters, but does nothing else. - - make coverage-baseline - - This captures baseline coverage information. - - make coverage-report - - This creates the coverage report. - - make coverage-clean-report - - This removes the generated coverage report without cleaning the cover- - age data itself. - - make coverage-clean-data - - This removes the captured coverage data without removing the coverage - files created at compile time (*.gcno). - - make coverage-clean - - This cleans all coverage data including the generated coverage report. - For more information about code coverage, see the gcov and lcov docu- - mentation. - - -SEE ALSO - - pcreapi(3), pcre16, pcre32, pcre_config(3). - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 12 May 2013 - Copyright (c) 1997-2013 University of Cambridge. ------------------------------------------------------------------------------- - - -PCREMATCHING(3) Library Functions Manual PCREMATCHING(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -PCRE MATCHING ALGORITHMS - - This document describes the two different algorithms that are available - in PCRE for matching a compiled regular expression against a given sub- - ject string. The "standard" algorithm is the one provided by the - pcre_exec(), pcre16_exec() and pcre32_exec() functions. These work in - the same as as Perl's matching function, and provide a Perl-compatible - matching operation. The just-in-time (JIT) optimization that is - described in the pcrejit documentation is compatible with these func- - tions. - - An alternative algorithm is provided by the pcre_dfa_exec(), - pcre16_dfa_exec() and pcre32_dfa_exec() functions; they operate in a - different way, and are not Perl-compatible. This alternative has advan- - tages and disadvantages compared with the standard algorithm, and these - are described below. - - When there is only one possible way in which a given subject string can - match a pattern, the two algorithms give the same answer. A difference - arises, however, when there are multiple possibilities. For example, if - the pattern - - ^<.*> - - is matched against the string - - <something> <something else> <something further> - - there are three possible answers. The standard algorithm finds only one - of them, whereas the alternative algorithm finds all three. - - -REGULAR EXPRESSIONS AS TREES - - The set of strings that are matched by a regular expression can be rep- - resented as a tree structure. An unlimited repetition in the pattern - makes the tree of infinite size, but it is still a tree. Matching the - pattern to a given subject string (from a given starting point) can be - thought of as a search of the tree. There are two ways to search a - tree: depth-first and breadth-first, and these correspond to the two - matching algorithms provided by PCRE. - - -THE STANDARD MATCHING ALGORITHM - - In the terminology of Jeffrey Friedl's book "Mastering Regular Expres- - sions", the standard algorithm is an "NFA algorithm". It conducts a - depth-first search of the pattern tree. That is, it proceeds along a - single path through the tree, checking that the subject matches what is - required. When there is a mismatch, the algorithm tries any alterna- - tives at the current point, and if they all fail, it backs up to the - previous branch point in the tree, and tries the next alternative - branch at that level. This often involves backing up (moving to the - left) in the subject string as well. The order in which repetition - branches are tried is controlled by the greedy or ungreedy nature of - the quantifier. - - If a leaf node is reached, a matching string has been found, and at - that point the algorithm stops. Thus, if there is more than one possi- - ble match, this algorithm returns the first one that it finds. Whether - this is the shortest, the longest, or some intermediate length depends - on the way the greedy and ungreedy repetition quantifiers are specified - in the pattern. - - Because it ends up with a single path through the tree, it is rela- - tively straightforward for this algorithm to keep track of the sub- - strings that are matched by portions of the pattern in parentheses. - This provides support for capturing parentheses and back references. - - -THE ALTERNATIVE MATCHING ALGORITHM - - This algorithm conducts a breadth-first search of the tree. Starting - from the first matching point in the subject, it scans the subject - string from left to right, once, character by character, and as it does - this, it remembers all the paths through the tree that represent valid - matches. In Friedl's terminology, this is a kind of "DFA algorithm", - though it is not implemented as a traditional finite state machine (it - keeps multiple states active simultaneously). - - Although the general principle of this matching algorithm is that it - scans the subject string only once, without backtracking, there is one - exception: when a lookaround assertion is encountered, the characters - following or preceding the current point have to be independently - inspected. - - The scan continues until either the end of the subject is reached, or - there are no more unterminated paths. At this point, terminated paths - represent the different matching possibilities (if there are none, the - match has failed). Thus, if there is more than one possible match, - this algorithm finds all of them, and in particular, it finds the long- - est. The matches are returned in decreasing order of length. There is - an option to stop the algorithm after the first match (which is neces- - sarily the shortest) is found. - - Note that all the matches that are found start at the same point in the - subject. If the pattern - - cat(er(pillar)?)? - - is matched against the string "the caterpillar catchment", the result - will be the three strings "caterpillar", "cater", and "cat" that start - at the fifth character of the subject. The algorithm does not automati- - cally move on to find matches that start at later positions. - - PCRE's "auto-possessification" optimization usually applies to charac- - ter repeats at the end of a pattern (as well as internally). For exam- - ple, the pattern "a\d+" is compiled as if it were "a\d++" because there - is no point even considering the possibility of backtracking into the - repeated digits. For DFA matching, this means that only one possible - match is found. If you really do want multiple matches in such cases, - either use an ungreedy repeat ("a\d+?") or set the PCRE_NO_AUTO_POSSESS - option when compiling. - - There are a number of features of PCRE regular expressions that are not - supported by the alternative matching algorithm. They are as follows: - - 1. Because the algorithm finds all possible matches, the greedy or - ungreedy nature of repetition quantifiers is not relevant. Greedy and - ungreedy quantifiers are treated in exactly the same way. However, pos- - sessive quantifiers can make a difference when what follows could also - match what is quantified, for example in a pattern like this: - - ^a++\w! - - This pattern matches "aaab!" but not "aaa!", which would be matched by - a non-possessive quantifier. Similarly, if an atomic group is present, - it is matched as if it were a standalone pattern at the current point, - and the longest match is then "locked in" for the rest of the overall - pattern. - - 2. When dealing with multiple paths through the tree simultaneously, it - is not straightforward to keep track of captured substrings for the - different matching possibilities, and PCRE's implementation of this - algorithm does not attempt to do this. This means that no captured sub- - strings are available. - - 3. Because no substrings are captured, back references within the pat- - tern are not supported, and cause errors if encountered. - - 4. For the same reason, conditional expressions that use a backrefer- - ence as the condition or test for a specific group recursion are not - supported. - - 5. Because many paths through the tree may be active, the \K escape - sequence, which resets the start of the match when encountered (but may - be on some paths and not on others), is not supported. It causes an - error if encountered. - - 6. Callouts are supported, but the value of the capture_top field is - always 1, and the value of the capture_last field is always -1. - - 7. The \C escape sequence, which (in the standard algorithm) always - matches a single data unit, even in UTF-8, UTF-16 or UTF-32 modes, is - not supported in these modes, because the alternative algorithm moves - through the subject string one character (not data unit) at a time, for - all active paths through the tree. - - 8. Except for (*FAIL), the backtracking control verbs such as (*PRUNE) - are not supported. (*FAIL) is supported, and behaves like a failing - negative assertion. - - -ADVANTAGES OF THE ALTERNATIVE ALGORITHM - - Using the alternative matching algorithm provides the following advan- - tages: - - 1. All possible matches (at a single point in the subject) are automat- - ically found, and in particular, the longest match is found. To find - more than one match using the standard algorithm, you have to do kludgy - things with callouts. - - 2. Because the alternative algorithm scans the subject string just - once, and never needs to backtrack (except for lookbehinds), it is pos- - sible to pass very long subject strings to the matching function in - several pieces, checking for partial matching each time. Although it is - possible to do multi-segment matching using the standard algorithm by - retaining partially matched substrings, it is more complicated. The - pcrepartial documentation gives details of partial matching and dis- - cusses multi-segment matching. - - -DISADVANTAGES OF THE ALTERNATIVE ALGORITHM - - The alternative algorithm suffers from a number of disadvantages: - - 1. It is substantially slower than the standard algorithm. This is - partly because it has to search for all possible matches, but is also - because it is less susceptible to optimization. - - 2. Capturing parentheses and back references are not supported. - - 3. Although atomic groups are supported, their use does not provide the - performance advantage that it does for the standard algorithm. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 12 November 2013 - Copyright (c) 1997-2012 University of Cambridge. ------------------------------------------------------------------------------- - - -PCREAPI(3) Library Functions Manual PCREAPI(3) - - - -NAME - PCRE - Perl-compatible regular expressions - - #include <pcre.h> - - -PCRE NATIVE API BASIC FUNCTIONS - - pcre *pcre_compile(const char *pattern, int options, - const char **errptr, int *erroffset, - const unsigned char *tableptr); - - pcre *pcre_compile2(const char *pattern, int options, - int *errorcodeptr, - const char **errptr, int *erroffset, - const unsigned char *tableptr); - - pcre_extra *pcre_study(const pcre *code, int options, - const char **errptr); - - void pcre_free_study(pcre_extra *extra); - - int pcre_exec(const pcre *code, const pcre_extra *extra, - const char *subject, int length, int startoffset, - int options, int *ovector, int ovecsize); - - int pcre_dfa_exec(const pcre *code, const pcre_extra *extra, - const char *subject, int length, int startoffset, - int options, int *ovector, int ovecsize, - int *workspace, int wscount); - - -PCRE NATIVE API STRING EXTRACTION FUNCTIONS - - int pcre_copy_named_substring(const pcre *code, - const char *subject, int *ovector, - int stringcount, const char *stringname, - char *buffer, int buffersize); - - int pcre_copy_substring(const char *subject, int *ovector, - int stringcount, int stringnumber, char *buffer, - int buffersize); - - int pcre_get_named_substring(const pcre *code, - const char *subject, int *ovector, - int stringcount, const char *stringname, - const char **stringptr); - - int pcre_get_stringnumber(const pcre *code, - const char *name); - - int pcre_get_stringtable_entries(const pcre *code, - const char *name, char **first, char **last); - - int pcre_get_substring(const char *subject, int *ovector, - int stringcount, int stringnumber, - const char **stringptr); - - int pcre_get_substring_list(const char *subject, - int *ovector, int stringcount, const char ***listptr); - - void pcre_free_substring(const char *stringptr); - - void pcre_free_substring_list(const char **stringptr); - - -PCRE NATIVE API AUXILIARY FUNCTIONS - - int pcre_jit_exec(const pcre *code, const pcre_extra *extra, - const char *subject, int length, int startoffset, - int options, int *ovector, int ovecsize, - pcre_jit_stack *jstack); - - pcre_jit_stack *pcre_jit_stack_alloc(int startsize, int maxsize); - - void pcre_jit_stack_free(pcre_jit_stack *stack); - - void pcre_assign_jit_stack(pcre_extra *extra, - pcre_jit_callback callback, void *data); - - const unsigned char *pcre_maketables(void); - - int pcre_fullinfo(const pcre *code, const pcre_extra *extra, - int what, void *where); - - int pcre_refcount(pcre *code, int adjust); - - int pcre_config(int what, void *where); - - const char *pcre_version(void); - - int pcre_pattern_to_host_byte_order(pcre *code, - pcre_extra *extra, const unsigned char *tables); - - -PCRE NATIVE API INDIRECTED FUNCTIONS - - void *(*pcre_malloc)(size_t); - - void (*pcre_free)(void *); - - void *(*pcre_stack_malloc)(size_t); - - void (*pcre_stack_free)(void *); - - int (*pcre_callout)(pcre_callout_block *); - - int (*pcre_stack_guard)(void); - - -PCRE 8-BIT, 16-BIT, AND 32-BIT LIBRARIES - - As well as support for 8-bit character strings, PCRE also supports - 16-bit strings (from release 8.30) and 32-bit strings (from release - 8.32), by means of two additional libraries. They can be built as well - as, or instead of, the 8-bit library. To avoid too much complication, - this document describes the 8-bit versions of the functions, with only - occasional references to the 16-bit and 32-bit libraries. - - The 16-bit and 32-bit functions operate in the same way as their 8-bit - counterparts; they just use different data types for their arguments - and results, and their names start with pcre16_ or pcre32_ instead of - pcre_. For every option that has UTF8 in its name (for example, - PCRE_UTF8), there are corresponding 16-bit and 32-bit names with UTF8 - replaced by UTF16 or UTF32, respectively. This facility is in fact just - cosmetic; the 16-bit and 32-bit option names define the same bit val- - ues. - - References to bytes and UTF-8 in this document should be read as refer- - ences to 16-bit data units and UTF-16 when using the 16-bit library, or - 32-bit data units and UTF-32 when using the 32-bit library, unless - specified otherwise. More details of the specific differences for the - 16-bit and 32-bit libraries are given in the pcre16 and pcre32 pages. - - -PCRE API OVERVIEW - - PCRE has its own native API, which is described in this document. There - are also some wrapper functions (for the 8-bit library only) that cor- - respond to the POSIX regular expression API, but they do not give - access to all the functionality. They are described in the pcreposix - documentation. Both of these APIs define a set of C function calls. A - C++ wrapper (again for the 8-bit library only) is also distributed with - PCRE. It is documented in the pcrecpp page. - - The native API C function prototypes are defined in the header file - pcre.h, and on Unix-like systems the (8-bit) library itself is called - libpcre. It can normally be accessed by adding -lpcre to the command - for linking an application that uses PCRE. The header file defines the - macros PCRE_MAJOR and PCRE_MINOR to contain the major and minor release - numbers for the library. Applications can use these to include support - for different releases of PCRE. - - In a Windows environment, if you want to statically link an application - program against a non-dll pcre.a file, you must define PCRE_STATIC - before including pcre.h or pcrecpp.h, because otherwise the pcre_mal- - loc() and pcre_free() exported functions will be declared - __declspec(dllimport), with unwanted results. - - The functions pcre_compile(), pcre_compile2(), pcre_study(), and - pcre_exec() are used for compiling and matching regular expressions in - a Perl-compatible manner. A sample program that demonstrates the sim- - plest way of using them is provided in the file called pcredemo.c in - the PCRE source distribution. A listing of this program is given in the - pcredemo documentation, and the pcresample documentation describes how - to compile and run it. - - Just-in-time compiler support is an optional feature of PCRE that can - be built in appropriate hardware environments. It greatly speeds up the - matching performance of many patterns. Simple programs can easily - request that it be used if available, by setting an option that is - ignored when it is not relevant. More complicated programs might need - to make use of the functions pcre_jit_stack_alloc(), - pcre_jit_stack_free(), and pcre_assign_jit_stack() in order to control - the JIT code's memory usage. - - From release 8.32 there is also a direct interface for JIT execution, - which gives improved performance. The JIT-specific functions are dis- - cussed in the pcrejit documentation. - - A second matching function, pcre_dfa_exec(), which is not Perl-compati- - ble, is also provided. This uses a different algorithm for the match- - ing. The alternative algorithm finds all possible matches (at a given - point in the subject), and scans the subject just once (unless there - are lookbehind assertions). However, this algorithm does not return - captured substrings. A description of the two matching algorithms and - their advantages and disadvantages is given in the pcrematching docu- - mentation. - - In addition to the main compiling and matching functions, there are - convenience functions for extracting captured substrings from a subject - string that is matched by pcre_exec(). They are: - - pcre_copy_substring() - pcre_copy_named_substring() - pcre_get_substring() - pcre_get_named_substring() - pcre_get_substring_list() - pcre_get_stringnumber() - pcre_get_stringtable_entries() - - pcre_free_substring() and pcre_free_substring_list() are also provided, - to free the memory used for extracted strings. - - The function pcre_maketables() is used to build a set of character - tables in the current locale for passing to pcre_compile(), - pcre_exec(), or pcre_dfa_exec(). This is an optional facility that is - provided for specialist use. Most commonly, no special tables are - passed, in which case internal tables that are generated when PCRE is - built are used. - - The function pcre_fullinfo() is used to find out information about a - compiled pattern. The function pcre_version() returns a pointer to a - string containing the version of PCRE and its date of release. - - The function pcre_refcount() maintains a reference count in a data - block containing a compiled pattern. This is provided for the benefit - of object-oriented applications. - - The global variables pcre_malloc and pcre_free initially contain the - entry points of the standard malloc() and free() functions, respec- - tively. PCRE calls the memory management functions via these variables, - so a calling program can replace them if it wishes to intercept the - calls. This should be done before calling any PCRE functions. - - The global variables pcre_stack_malloc and pcre_stack_free are also - indirections to memory management functions. These special functions - are used only when PCRE is compiled to use the heap for remembering - data, instead of recursive function calls, when running the pcre_exec() - function. See the pcrebuild documentation for details of how to do - this. It is a non-standard way of building PCRE, for use in environ- - ments that have limited stacks. Because of the greater use of memory - management, it runs more slowly. Separate functions are provided so - that special-purpose external code can be used for this case. When - used, these functions are always called in a stack-like manner (last - obtained, first freed), and always for memory blocks of the same size. - There is a discussion about PCRE's stack usage in the pcrestack docu- - mentation. - - The global variable pcre_callout initially contains NULL. It can be set - by the caller to a "callout" function, which PCRE will then call at - specified points during a matching operation. Details are given in the - pcrecallout documentation. - - The global variable pcre_stack_guard initially contains NULL. It can be - set by the caller to a function that is called by PCRE whenever it - starts to compile a parenthesized part of a pattern. When parentheses - are nested, PCRE uses recursive function calls, which use up the system - stack. This function is provided so that applications with restricted - stacks can force a compilation error if the stack runs out. The func- - tion should return zero if all is well, or non-zero to force an error. - - -NEWLINES - - PCRE supports five different conventions for indicating line breaks in - strings: a single CR (carriage return) character, a single LF (line- - feed) character, the two-character sequence CRLF, any of the three pre- - ceding, or any Unicode newline sequence. The Unicode newline sequences - are the three just mentioned, plus the single characters VT (vertical - tab, U+000B), FF (form feed, U+000C), NEL (next line, U+0085), LS (line - separator, U+2028), and PS (paragraph separator, U+2029). - - Each of the first three conventions is used by at least one operating - system as its standard newline sequence. When PCRE is built, a default - can be specified. The default default is LF, which is the Unix stan- - dard. When PCRE is run, the default can be overridden, either when a - pattern is compiled, or when it is matched. - - At compile time, the newline convention can be specified by the options - argument of pcre_compile(), or it can be specified by special text at - the start of the pattern itself; this overrides any other settings. See - the pcrepattern page for details of the special character sequences. - - In the PCRE documentation the word "newline" is used to mean "the char- - acter or pair of characters that indicate a line break". The choice of - newline convention affects the handling of the dot, circumflex, and - dollar metacharacters, the handling of #-comments in /x mode, and, when - CRLF is a recognized line ending sequence, the match position advance- - ment for a non-anchored pattern. There is more detail about this in the - section on pcre_exec() options below. - - The choice of newline convention does not affect the interpretation of - the \n or \r escape sequences, nor does it affect what \R matches, - which is controlled in a similar way, but by separate options. - - -MULTITHREADING - - The PCRE functions can be used in multi-threading applications, with - the proviso that the memory management functions pointed to by - pcre_malloc, pcre_free, pcre_stack_malloc, and pcre_stack_free, and the - callout and stack-checking functions pointed to by pcre_callout and - pcre_stack_guard, are shared by all threads. - - The compiled form of a regular expression is not altered during match- - ing, so the same compiled pattern can safely be used by several threads - at once. - - If the just-in-time optimization feature is being used, it needs sepa- - rate memory stack areas for each thread. See the pcrejit documentation - for more details. - - -SAVING PRECOMPILED PATTERNS FOR LATER USE - - The compiled form of a regular expression can be saved and re-used at a - later time, possibly by a different program, and even on a host other - than the one on which it was compiled. Details are given in the - pcreprecompile documentation, which includes a description of the - pcre_pattern_to_host_byte_order() function. However, compiling a regu- - lar expression with one version of PCRE for use with a different ver- - sion is not guaranteed to work and may cause crashes. - - -CHECKING BUILD-TIME OPTIONS - - int pcre_config(int what, void *where); - - The function pcre_config() makes it possible for a PCRE client to dis- - cover which optional features have been compiled into the PCRE library. - The pcrebuild documentation has more details about these optional fea- - tures. - - The first argument for pcre_config() is an integer, specifying which - information is required; the second argument is a pointer to a variable - into which the information is placed. The returned value is zero on - success, or the negative error code PCRE_ERROR_BADOPTION if the value - in the first argument is not recognized. The following information is - available: - - PCRE_CONFIG_UTF8 - - The output is an integer that is set to one if UTF-8 support is avail- - able; otherwise it is set to zero. This value should normally be given - to the 8-bit version of this function, pcre_config(). If it is given to - the 16-bit or 32-bit version of this function, the result is - PCRE_ERROR_BADOPTION. - - PCRE_CONFIG_UTF16 - - The output is an integer that is set to one if UTF-16 support is avail- - able; otherwise it is set to zero. This value should normally be given - to the 16-bit version of this function, pcre16_config(). If it is given - to the 8-bit or 32-bit version of this function, the result is - PCRE_ERROR_BADOPTION. - - PCRE_CONFIG_UTF32 - - The output is an integer that is set to one if UTF-32 support is avail- - able; otherwise it is set to zero. This value should normally be given - to the 32-bit version of this function, pcre32_config(). If it is given - to the 8-bit or 16-bit version of this function, the result is - PCRE_ERROR_BADOPTION. - - PCRE_CONFIG_UNICODE_PROPERTIES - - The output is an integer that is set to one if support for Unicode - character properties is available; otherwise it is set to zero. - - PCRE_CONFIG_JIT - - The output is an integer that is set to one if support for just-in-time - compiling is available; otherwise it is set to zero. - - PCRE_CONFIG_JITTARGET - - The output is a pointer to a zero-terminated "const char *" string. If - JIT support is available, the string contains the name of the architec- - ture for which the JIT compiler is configured, for example "x86 32bit - (little endian + unaligned)". If JIT support is not available, the - result is NULL. - - PCRE_CONFIG_NEWLINE - - The output is an integer whose value specifies the default character - sequence that is recognized as meaning "newline". The values that are - supported in ASCII/Unicode environments are: 10 for LF, 13 for CR, 3338 - for CRLF, -2 for ANYCRLF, and -1 for ANY. In EBCDIC environments, CR, - ANYCRLF, and ANY yield the same values. However, the value for LF is - normally 21, though some EBCDIC environments use 37. The corresponding - values for CRLF are 3349 and 3365. The default should normally corre- - spond to the standard sequence for your operating system. - - PCRE_CONFIG_BSR - - The output is an integer whose value indicates what character sequences - the \R escape sequence matches by default. A value of 0 means that \R - matches any Unicode line ending sequence; a value of 1 means that \R - matches only CR, LF, or CRLF. The default can be overridden when a pat- - tern is compiled or matched. - - PCRE_CONFIG_LINK_SIZE - - The output is an integer that contains the number of bytes used for - internal linkage in compiled regular expressions. For the 8-bit - library, the value can be 2, 3, or 4. For the 16-bit library, the value - is either 2 or 4 and is still a number of bytes. For the 32-bit - library, the value is either 2 or 4 and is still a number of bytes. The - default value of 2 is sufficient for all but the most massive patterns, - since it allows the compiled pattern to be up to 64K in size. Larger - values allow larger regular expressions to be compiled, at the expense - of slower matching. - - PCRE_CONFIG_POSIX_MALLOC_THRESHOLD - - The output is an integer that contains the threshold above which the - POSIX interface uses malloc() for output vectors. Further details are - given in the pcreposix documentation. - - PCRE_CONFIG_PARENS_LIMIT - - The output is a long integer that gives the maximum depth of nesting of - parentheses (of any kind) in a pattern. This limit is imposed to cap - the amount of system stack used when a pattern is compiled. It is spec- - ified when PCRE is built; the default is 250. This limit does not take - into account the stack that may already be used by the calling applica- - tion. For finer control over compilation stack usage, you can set a - pointer to an external checking function in pcre_stack_guard. - - PCRE_CONFIG_MATCH_LIMIT - - The output is a long integer that gives the default limit for the num- - ber of internal matching function calls in a pcre_exec() execution. - Further details are given with pcre_exec() below. - - PCRE_CONFIG_MATCH_LIMIT_RECURSION - - The output is a long integer that gives the default limit for the depth - of recursion when calling the internal matching function in a - pcre_exec() execution. Further details are given with pcre_exec() - below. - - PCRE_CONFIG_STACKRECURSE - - The output is an integer that is set to one if internal recursion when - running pcre_exec() is implemented by recursive function calls that use - the stack to remember their state. This is the usual way that PCRE is - compiled. The output is zero if PCRE was compiled to use blocks of data - on the heap instead of recursive function calls. In this case, - pcre_stack_malloc and pcre_stack_free are called to manage memory - blocks on the heap, thus avoiding the use of the stack. - - -COMPILING A PATTERN - - pcre *pcre_compile(const char *pattern, int options, - const char **errptr, int *erroffset, - const unsigned char *tableptr); - - pcre *pcre_compile2(const char *pattern, int options, - int *errorcodeptr, - const char **errptr, int *erroffset, - const unsigned char *tableptr); - - Either of the functions pcre_compile() or pcre_compile2() can be called - to compile a pattern into an internal form. The only difference between - the two interfaces is that pcre_compile2() has an additional argument, - errorcodeptr, via which a numerical error code can be returned. To - avoid too much repetition, we refer just to pcre_compile() below, but - the information applies equally to pcre_compile2(). - - The pattern is a C string terminated by a binary zero, and is passed in - the pattern argument. A pointer to a single block of memory that is - obtained via pcre_malloc is returned. This contains the compiled code - and related data. The pcre type is defined for the returned block; this - is a typedef for a structure whose contents are not externally defined. - It is up to the caller to free the memory (via pcre_free) when it is no - longer required. - - Although the compiled code of a PCRE regex is relocatable, that is, it - does not depend on memory location, the complete pcre data block is not - fully relocatable, because it may contain a copy of the tableptr argu- - ment, which is an address (see below). - - The options argument contains various bit settings that affect the com- - pilation. It should be zero if no options are required. The available - options are described below. Some of them (in particular, those that - are compatible with Perl, but some others as well) can also be set and - unset from within the pattern (see the detailed description in the - pcrepattern documentation). For those options that can be different in - different parts of the pattern, the contents of the options argument - specifies their settings at the start of compilation and execution. The - PCRE_ANCHORED, PCRE_BSR_xxx, PCRE_NEWLINE_xxx, PCRE_NO_UTF8_CHECK, and - PCRE_NO_START_OPTIMIZE options can be set at the time of matching as - well as at compile time. - - If errptr is NULL, pcre_compile() returns NULL immediately. Otherwise, - if compilation of a pattern fails, pcre_compile() returns NULL, and - sets the variable pointed to by errptr to point to a textual error mes- - sage. This is a static string that is part of the library. You must not - try to free it. Normally, the offset from the start of the pattern to - the data unit that was being processed when the error was discovered is - placed in the variable pointed to by erroffset, which must not be NULL - (if it is, an immediate error is given). However, for an invalid UTF-8 - or UTF-16 string, the offset is that of the first data unit of the - failing character. - - Some errors are not detected until the whole pattern has been scanned; - in these cases, the offset passed back is the length of the pattern. - Note that the offset is in data units, not characters, even in a UTF - mode. It may sometimes point into the middle of a UTF-8 or UTF-16 char- - acter. - - If pcre_compile2() is used instead of pcre_compile(), and the error- - codeptr argument is not NULL, a non-zero error code number is returned - via this argument in the event of an error. This is in addition to the - textual error message. Error codes and messages are listed below. - - If the final argument, tableptr, is NULL, PCRE uses a default set of - character tables that are built when PCRE is compiled, using the - default C locale. Otherwise, tableptr must be an address that is the - result of a call to pcre_maketables(). This value is stored with the - compiled pattern, and used again by pcre_exec() and pcre_dfa_exec() - when the pattern is matched. For more discussion, see the section on - locale support below. - - This code fragment shows a typical straightforward call to pcre_com- - pile(): - - pcre *re; - const char *error; - int erroffset; - re = pcre_compile( - "^A.*Z", /* the pattern */ - 0, /* default options */ - &error, /* for error message */ - &erroffset, /* for error offset */ - NULL); /* use default character tables */ - - The following names for option bits are defined in the pcre.h header - file: - - PCRE_ANCHORED - - If this bit is set, the pattern is forced to be "anchored", that is, it - is constrained to match only at the first matching point in the string - that is being searched (the "subject string"). This effect can also be - achieved by appropriate constructs in the pattern itself, which is the - only way to do it in Perl. - - PCRE_AUTO_CALLOUT - - If this bit is set, pcre_compile() automatically inserts callout items, - all with number 255, before each pattern item. For discussion of the - callout facility, see the pcrecallout documentation. - - PCRE_BSR_ANYCRLF - PCRE_BSR_UNICODE - - These options (which are mutually exclusive) control what the \R escape - sequence matches. The choice is either to match only CR, LF, or CRLF, - or to match any Unicode newline sequence. The default is specified when - PCRE is built. It can be overridden from within the pattern, or by set- - ting an option when a compiled pattern is matched. - - PCRE_CASELESS - - If this bit is set, letters in the pattern match both upper and lower - case letters. It is equivalent to Perl's /i option, and it can be - changed within a pattern by a (?i) option setting. In UTF-8 mode, PCRE - always understands the concept of case for characters whose values are - less than 128, so caseless matching is always possible. For characters - with higher values, the concept of case is supported if PCRE is com- - piled with Unicode property support, but not otherwise. If you want to - use caseless matching for characters 128 and above, you must ensure - that PCRE is compiled with Unicode property support as well as with - UTF-8 support. - - PCRE_DOLLAR_ENDONLY - - If this bit is set, a dollar metacharacter in the pattern matches only - at the end of the subject string. Without this option, a dollar also - matches immediately before a newline at the end of the string (but not - before any other newlines). The PCRE_DOLLAR_ENDONLY option is ignored - if PCRE_MULTILINE is set. There is no equivalent to this option in - Perl, and no way to set it within a pattern. - - PCRE_DOTALL - - If this bit is set, a dot metacharacter in the pattern matches a char- - acter of any value, including one that indicates a newline. However, it - only ever matches one character, even if newlines are coded as CRLF. - Without this option, a dot does not match when the current position is - at a newline. This option is equivalent to Perl's /s option, and it can - be changed within a pattern by a (?s) option setting. A negative class - such as [^a] always matches newline characters, independent of the set- - ting of this option. - - PCRE_DUPNAMES - - If this bit is set, names used to identify capturing subpatterns need - not be unique. This can be helpful for certain types of pattern when it - is known that only one instance of the named subpattern can ever be - matched. There are more details of named subpatterns below; see also - the pcrepattern documentation. - - PCRE_EXTENDED - - If this bit is set, most white space characters in the pattern are - totally ignored except when escaped or inside a character class. How- - ever, white space is not allowed within sequences such as (?> that - introduce various parenthesized subpatterns, nor within a numerical - quantifier such as {1,3}. However, ignorable white space is permitted - between an item and a following quantifier and between a quantifier and - a following + that indicates possessiveness. - - White space did not used to include the VT character (code 11), because - Perl did not treat this character as white space. However, Perl changed - at release 5.18, so PCRE followed at release 8.34, and VT is now - treated as white space. - - PCRE_EXTENDED also causes characters between an unescaped # outside a - character class and the next newline, inclusive, to be ignored. - PCRE_EXTENDED is equivalent to Perl's /x option, and it can be changed - within a pattern by a (?x) option setting. - - Which characters are interpreted as newlines is controlled by the - options passed to pcre_compile() or by a special sequence at the start - of the pattern, as described in the section entitled "Newline conven- - tions" in the pcrepattern documentation. Note that the end of this type - of comment is a literal newline sequence in the pattern; escape - sequences that happen to represent a newline do not count. - - This option makes it possible to include comments inside complicated - patterns. Note, however, that this applies only to data characters. - White space characters may never appear within special character - sequences in a pattern, for example within the sequence (?( that intro- - duces a conditional subpattern. - - PCRE_EXTRA - - This option was invented in order to turn on additional functionality - of PCRE that is incompatible with Perl, but it is currently of very - little use. When set, any backslash in a pattern that is followed by a - letter that has no special meaning causes an error, thus reserving - these combinations for future expansion. By default, as in Perl, a - backslash followed by a letter with no special meaning is treated as a - literal. (Perl can, however, be persuaded to give an error for this, by - running it with the -w option.) There are at present no other features - controlled by this option. It can also be set by a (?X) option setting - within a pattern. - - PCRE_FIRSTLINE - - If this option is set, an unanchored pattern is required to match - before or at the first newline in the subject string, though the - matched text may continue over the newline. - - PCRE_JAVASCRIPT_COMPAT - - If this option is set, PCRE's behaviour is changed in some ways so that - it is compatible with JavaScript rather than Perl. The changes are as - follows: - - (1) A lone closing square bracket in a pattern causes a compile-time - error, because this is illegal in JavaScript (by default it is treated - as a data character). Thus, the pattern AB]CD becomes illegal when this - option is set. - - (2) At run time, a back reference to an unset subpattern group matches - an empty string (by default this causes the current matching alterna- - tive to fail). A pattern such as (\1)(a) succeeds when this option is - set (assuming it can find an "a" in the subject), whereas it fails by - default, for Perl compatibility. - - (3) \U matches an upper case "U" character; by default \U causes a com- - pile time error (Perl uses \U to upper case subsequent characters). - - (4) \u matches a lower case "u" character unless it is followed by four - hexadecimal digits, in which case the hexadecimal number defines the - code point to match. By default, \u causes a compile time error (Perl - uses it to upper case the following character). - - (5) \x matches a lower case "x" character unless it is followed by two - hexadecimal digits, in which case the hexadecimal number defines the - code point to match. By default, as in Perl, a hexadecimal number is - always expected after \x, but it may have zero, one, or two digits (so, - for example, \xz matches a binary zero character followed by z). - - PCRE_MULTILINE - - By default, for the purposes of matching "start of line" and "end of - line", PCRE treats the subject string as consisting of a single line of - characters, even if it actually contains newlines. The "start of line" - metacharacter (^) matches only at the start of the string, and the "end - of line" metacharacter ($) matches only at the end of the string, or - before a terminating newline (except when PCRE_DOLLAR_ENDONLY is set). - Note, however, that unless PCRE_DOTALL is set, the "any character" - metacharacter (.) does not match at a newline. This behaviour (for ^, - $, and dot) is the same as Perl. - - When PCRE_MULTILINE it is set, the "start of line" and "end of line" - constructs match immediately following or immediately before internal - newlines in the subject string, respectively, as well as at the very - start and end. This is equivalent to Perl's /m option, and it can be - changed within a pattern by a (?m) option setting. If there are no new- - lines in a subject string, or no occurrences of ^ or $ in a pattern, - setting PCRE_MULTILINE has no effect. - - PCRE_NEVER_UTF - - This option locks out interpretation of the pattern as UTF-8 (or UTF-16 - or UTF-32 in the 16-bit and 32-bit libraries). In particular, it pre- - vents the creator of the pattern from switching to UTF interpretation - by starting the pattern with (*UTF). This may be useful in applications - that process patterns from external sources. The combination of - PCRE_UTF8 and PCRE_NEVER_UTF also causes an error. - - PCRE_NEWLINE_CR - PCRE_NEWLINE_LF - PCRE_NEWLINE_CRLF - PCRE_NEWLINE_ANYCRLF - PCRE_NEWLINE_ANY - - These options override the default newline definition that was chosen - when PCRE was built. Setting the first or the second specifies that a - newline is indicated by a single character (CR or LF, respectively). - Setting PCRE_NEWLINE_CRLF specifies that a newline is indicated by the - two-character CRLF sequence. Setting PCRE_NEWLINE_ANYCRLF specifies - that any of the three preceding sequences should be recognized. Setting - PCRE_NEWLINE_ANY specifies that any Unicode newline sequence should be - recognized. - - In an ASCII/Unicode environment, the Unicode newline sequences are the - three just mentioned, plus the single characters VT (vertical tab, - U+000B), FF (form feed, U+000C), NEL (next line, U+0085), LS (line sep- - arator, U+2028), and PS (paragraph separator, U+2029). For the 8-bit - library, the last two are recognized only in UTF-8 mode. - - When PCRE is compiled to run in an EBCDIC (mainframe) environment, the - code for CR is 0x0d, the same as ASCII. However, the character code for - LF is normally 0x15, though in some EBCDIC environments 0x25 is used. - Whichever of these is not LF is made to correspond to Unicode's NEL - character. EBCDIC codes are all less than 256. For more details, see - the pcrebuild documentation. - - The newline setting in the options word uses three bits that are - treated as a number, giving eight possibilities. Currently only six are - used (default plus the five values above). This means that if you set - more than one newline option, the combination may or may not be sensi- - ble. For example, PCRE_NEWLINE_CR with PCRE_NEWLINE_LF is equivalent to - PCRE_NEWLINE_CRLF, but other combinations may yield unused numbers and - cause an error. - - The only time that a line break in a pattern is specially recognized - when compiling is when PCRE_EXTENDED is set. CR and LF are white space - characters, and so are ignored in this mode. Also, an unescaped # out- - side a character class indicates a comment that lasts until after the - next line break sequence. In other circumstances, line break sequences - in patterns are treated as literal data. - - The newline option that is set at compile time becomes the default that - is used for pcre_exec() and pcre_dfa_exec(), but it can be overridden. - - PCRE_NO_AUTO_CAPTURE - - If this option is set, it disables the use of numbered capturing paren- - theses in the pattern. Any opening parenthesis that is not followed by - ? behaves as if it were followed by ?: but named parentheses can still - be used for capturing (and they acquire numbers in the usual way). - There is no equivalent of this option in Perl. - - PCRE_NO_AUTO_POSSESS - - If this option is set, it disables "auto-possessification". This is an - optimization that, for example, turns a+b into a++b in order to avoid - backtracks into a+ that can never be successful. However, if callouts - are in use, auto-possessification means that some of them are never - taken. You can set this option if you want the matching functions to do - a full unoptimized search and run all the callouts, but it is mainly - provided for testing purposes. - - PCRE_NO_START_OPTIMIZE - - This is an option that acts at matching time; that is, it is really an - option for pcre_exec() or pcre_dfa_exec(). If it is set at compile - time, it is remembered with the compiled pattern and assumed at match- - ing time. This is necessary if you want to use JIT execution, because - the JIT compiler needs to know whether or not this option is set. For - details see the discussion of PCRE_NO_START_OPTIMIZE below. - - PCRE_UCP - - This option changes the way PCRE processes \B, \b, \D, \d, \S, \s, \W, - \w, and some of the POSIX character classes. By default, only ASCII - characters are recognized, but if PCRE_UCP is set, Unicode properties - are used instead to classify characters. More details are given in the - section on generic character types in the pcrepattern page. If you set - PCRE_UCP, matching one of the items it affects takes much longer. The - option is available only if PCRE has been compiled with Unicode prop- - erty support. - - PCRE_UNGREEDY - - This option inverts the "greediness" of the quantifiers so that they - are not greedy by default, but become greedy if followed by "?". It is - not compatible with Perl. It can also be set by a (?U) option setting - within the pattern. - - PCRE_UTF8 - - This option causes PCRE to regard both the pattern and the subject as - strings of UTF-8 characters instead of single-byte strings. However, it - is available only when PCRE is built to include UTF support. If not, - the use of this option provokes an error. Details of how this option - changes the behaviour of PCRE are given in the pcreunicode page. - - PCRE_NO_UTF8_CHECK - - When PCRE_UTF8 is set, the validity of the pattern as a UTF-8 string is - automatically checked. There is a discussion about the validity of - UTF-8 strings in the pcreunicode page. If an invalid UTF-8 sequence is - found, pcre_compile() returns an error. If you already know that your - pattern is valid, and you want to skip this check for performance rea- - sons, you can set the PCRE_NO_UTF8_CHECK option. When it is set, the - effect of passing an invalid UTF-8 string as a pattern is undefined. It - may cause your program to crash or loop. Note that this option can also - be passed to pcre_exec() and pcre_dfa_exec(), to suppress the validity - checking of subject strings only. If the same string is being matched - many times, the option can be safely set for the second and subsequent - matchings to improve performance. - - -COMPILATION ERROR CODES - - The following table lists the error codes than may be returned by - pcre_compile2(), along with the error messages that may be returned by - both compiling functions. Note that error messages are always 8-bit - ASCII strings, even in 16-bit or 32-bit mode. As PCRE has developed, - some error codes have fallen out of use. To avoid confusion, they have - not been re-used. - - 0 no error - 1 \ at end of pattern - 2 \c at end of pattern - 3 unrecognized character follows \ - 4 numbers out of order in {} quantifier - 5 number too big in {} quantifier - 6 missing terminating ] for character class - 7 invalid escape sequence in character class - 8 range out of order in character class - 9 nothing to repeat - 10 [this code is not in use] - 11 internal error: unexpected repeat - 12 unrecognized character after (? or (?- - 13 POSIX named classes are supported only within a class - 14 missing ) - 15 reference to non-existent subpattern - 16 erroffset passed as NULL - 17 unknown option bit(s) set - 18 missing ) after comment - 19 [this code is not in use] - 20 regular expression is too large - 21 failed to get memory - 22 unmatched parentheses - 23 internal error: code overflow - 24 unrecognized character after (?< - 25 lookbehind assertion is not fixed length - 26 malformed number or name after (?( - 27 conditional group contains more than two branches - 28 assertion expected after (?( - 29 (?R or (?[+-]digits must be followed by ) - 30 unknown POSIX class name - 31 POSIX collating elements are not supported - 32 this version of PCRE is compiled without UTF support - 33 [this code is not in use] - 34 character value in \x{} or \o{} is too large - 35 invalid condition (?(0) - 36 \C not allowed in lookbehind assertion - 37 PCRE does not support \L, \l, \N{name}, \U, or \u - 38 number after (?C is > 255 - 39 closing ) for (?C expected - 40 recursive call could loop indefinitely - 41 unrecognized character after (?P - 42 syntax error in subpattern name (missing terminator) - 43 two named subpatterns have the same name - 44 invalid UTF-8 string (specifically UTF-8) - 45 support for \P, \p, and \X has not been compiled - 46 malformed \P or \p sequence - 47 unknown property name after \P or \p - 48 subpattern name is too long (maximum 32 characters) - 49 too many named subpatterns (maximum 10000) - 50 [this code is not in use] - 51 octal value is greater than \377 in 8-bit non-UTF-8 mode - 52 internal error: overran compiling workspace - 53 internal error: previously-checked referenced subpattern - not found - 54 DEFINE group contains more than one branch - 55 repeating a DEFINE group is not allowed - 56 inconsistent NEWLINE options - 57 \g is not followed by a braced, angle-bracketed, or quoted - name/number or by a plain number - 58 a numbered reference must not be zero - 59 an argument is not allowed for (*ACCEPT), (*FAIL), or (*COMMIT) - 60 (*VERB) not recognized or malformed - 61 number is too big - 62 subpattern name expected - 63 digit expected after (?+ - 64 ] is an invalid data character in JavaScript compatibility mode - 65 different names for subpatterns of the same number are - not allowed - 66 (*MARK) must have an argument - 67 this version of PCRE is not compiled with Unicode property - support - 68 \c must be followed by an ASCII character - 69 \k is not followed by a braced, angle-bracketed, or quoted name - 70 internal error: unknown opcode in find_fixedlength() - 71 \N is not supported in a class - 72 too many forward references - 73 disallowed Unicode code point (>= 0xd800 && <= 0xdfff) - 74 invalid UTF-16 string (specifically UTF-16) - 75 name is too long in (*MARK), (*PRUNE), (*SKIP), or (*THEN) - 76 character value in \u.... sequence is too large - 77 invalid UTF-32 string (specifically UTF-32) - 78 setting UTF is disabled by the application - 79 non-hex character in \x{} (closing brace missing?) - 80 non-octal character in \o{} (closing brace missing?) - 81 missing opening brace after \o - 82 parentheses are too deeply nested - 83 invalid range in character class - 84 group name must start with a non-digit - 85 parentheses are too deeply nested (stack check) - - The numbers 32 and 10000 in errors 48 and 49 are defaults; different - values may be used if the limits were changed when PCRE was built. - - -STUDYING A PATTERN - - pcre_extra *pcre_study(const pcre *code, int options, - const char **errptr); - - If a compiled pattern is going to be used several times, it is worth - spending more time analyzing it in order to speed up the time taken for - matching. The function pcre_study() takes a pointer to a compiled pat- - tern as its first argument. If studying the pattern produces additional - information that will help speed up matching, pcre_study() returns a - pointer to a pcre_extra block, in which the study_data field points to - the results of the study. - - The returned value from pcre_study() can be passed directly to - pcre_exec() or pcre_dfa_exec(). However, a pcre_extra block also con- - tains other fields that can be set by the caller before the block is - passed; these are described below in the section on matching a pattern. - - If studying the pattern does not produce any useful information, - pcre_study() returns NULL by default. In that circumstance, if the - calling program wants to pass any of the other fields to pcre_exec() or - pcre_dfa_exec(), it must set up its own pcre_extra block. However, if - pcre_study() is called with the PCRE_STUDY_EXTRA_NEEDED option, it - returns a pcre_extra block even if studying did not find any additional - information. It may still return NULL, however, if an error occurs in - pcre_study(). - - The second argument of pcre_study() contains option bits. There are - three further options in addition to PCRE_STUDY_EXTRA_NEEDED: - - PCRE_STUDY_JIT_COMPILE - PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE - PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE - - If any of these are set, and the just-in-time compiler is available, - the pattern is further compiled into machine code that executes much - faster than the pcre_exec() interpretive matching function. If the - just-in-time compiler is not available, these options are ignored. All - undefined bits in the options argument must be zero. - - JIT compilation is a heavyweight optimization. It can take some time - for patterns to be analyzed, and for one-off matches and simple pat- - terns the benefit of faster execution might be offset by a much slower - study time. Not all patterns can be optimized by the JIT compiler. For - those that cannot be handled, matching automatically falls back to the - pcre_exec() interpreter. For more details, see the pcrejit documenta- - tion. - - The third argument for pcre_study() is a pointer for an error message. - If studying succeeds (even if no data is returned), the variable it - points to is set to NULL. Otherwise it is set to point to a textual - error message. This is a static string that is part of the library. You - must not try to free it. You should test the error pointer for NULL - after calling pcre_study(), to be sure that it has run successfully. - - When you are finished with a pattern, you can free the memory used for - the study data by calling pcre_free_study(). This function was added to - the API for release 8.20. For earlier versions, the memory could be - freed with pcre_free(), just like the pattern itself. This will still - work in cases where JIT optimization is not used, but it is advisable - to change to the new function when convenient. - - This is a typical way in which pcre_study() is used (except that in a - real application there should be tests for errors): - - int rc; - pcre *re; - pcre_extra *sd; - re = pcre_compile("pattern", 0, &error, &erroroffset, NULL); - sd = pcre_study( - re, /* result of pcre_compile() */ - 0, /* no options */ - &error); /* set to NULL or points to a message */ - rc = pcre_exec( /* see below for details of pcre_exec() options */ - re, sd, "subject", 7, 0, 0, ovector, 30); - ... - pcre_free_study(sd); - pcre_free(re); - - Studying a pattern does two things: first, a lower bound for the length - of subject string that is needed to match the pattern is computed. This - does not mean that there are any strings of that length that match, but - it does guarantee that no shorter strings match. The value is used to - avoid wasting time by trying to match strings that are shorter than the - lower bound. You can find out the value in a calling program via the - pcre_fullinfo() function. - - Studying a pattern is also useful for non-anchored patterns that do not - have a single fixed starting character. A bitmap of possible starting - bytes is created. This speeds up finding a position in the subject at - which to start matching. (In 16-bit mode, the bitmap is used for 16-bit - values less than 256. In 32-bit mode, the bitmap is used for 32-bit - values less than 256.) - - These two optimizations apply to both pcre_exec() and pcre_dfa_exec(), - and the information is also used by the JIT compiler. The optimiza- - tions can be disabled by setting the PCRE_NO_START_OPTIMIZE option. - You might want to do this if your pattern contains callouts or (*MARK) - and you want to make use of these facilities in cases where matching - fails. - - PCRE_NO_START_OPTIMIZE can be specified at either compile time or exe- - cution time. However, if PCRE_NO_START_OPTIMIZE is passed to - pcre_exec(), (that is, after any JIT compilation has happened) JIT exe- - cution is disabled. For JIT execution to work with PCRE_NO_START_OPTI- - MIZE, the option must be set at compile time. - - There is a longer discussion of PCRE_NO_START_OPTIMIZE below. - - -LOCALE SUPPORT - - PCRE handles caseless matching, and determines whether characters are - letters, digits, or whatever, by reference to a set of tables, indexed - by character code point. When running in UTF-8 mode, or in the 16- or - 32-bit libraries, this applies only to characters with code points less - than 256. By default, higher-valued code points never match escapes - such as \w or \d. However, if PCRE is built with Unicode property sup- - port, all characters can be tested with \p and \P, or, alternatively, - the PCRE_UCP option can be set when a pattern is compiled; this causes - \w and friends to use Unicode property support instead of the built-in - tables. - - The use of locales with Unicode is discouraged. If you are handling - characters with code points greater than 128, you should either use - Unicode support, or use locales, but not try to mix the two. - - PCRE contains an internal set of tables that are used when the final - argument of pcre_compile() is NULL. These are sufficient for many - applications. Normally, the internal tables recognize only ASCII char- - acters. However, when PCRE is built, it is possible to cause the inter- - nal tables to be rebuilt in the default "C" locale of the local system, - which may cause them to be different. - - The internal tables can always be overridden by tables supplied by the - application that calls PCRE. These may be created in a different locale - from the default. As more and more applications change to using Uni- - code, the need for this locale support is expected to die away. - - External tables are built by calling the pcre_maketables() function, - which has no arguments, in the relevant locale. The result can then be - passed to pcre_compile() as often as necessary. For example, to build - and use tables that are appropriate for the French locale (where - accented characters with values greater than 128 are treated as let- - ters), the following code could be used: - - setlocale(LC_CTYPE, "fr_FR"); - tables = pcre_maketables(); - re = pcre_compile(..., tables); - - The locale name "fr_FR" is used on Linux and other Unix-like systems; - if you are using Windows, the name for the French locale is "french". - - When pcre_maketables() runs, the tables are built in memory that is - obtained via pcre_malloc. It is the caller's responsibility to ensure - that the memory containing the tables remains available for as long as - it is needed. - - The pointer that is passed to pcre_compile() is saved with the compiled - pattern, and the same tables are used via this pointer by pcre_study() - and also by pcre_exec() and pcre_dfa_exec(). Thus, for any single pat- - tern, compilation, studying and matching all happen in the same locale, - but different patterns can be processed in different locales. - - It is possible to pass a table pointer or NULL (indicating the use of - the internal tables) to pcre_exec() or pcre_dfa_exec() (see the discus- - sion below in the section on matching a pattern). This facility is pro- - vided for use with pre-compiled patterns that have been saved and - reloaded. Character tables are not saved with patterns, so if a non- - standard table was used at compile time, it must be provided again when - the reloaded pattern is matched. Attempting to use this facility to - match a pattern in a different locale from the one in which it was com- - piled is likely to lead to anomalous (usually incorrect) results. - - -INFORMATION ABOUT A PATTERN - - int pcre_fullinfo(const pcre *code, const pcre_extra *extra, - int what, void *where); - - The pcre_fullinfo() function returns information about a compiled pat- - tern. It replaces the pcre_info() function, which was removed from the - library at version 8.30, after more than 10 years of obsolescence. - - The first argument for pcre_fullinfo() is a pointer to the compiled - pattern. The second argument is the result of pcre_study(), or NULL if - the pattern was not studied. The third argument specifies which piece - of information is required, and the fourth argument is a pointer to a - variable to receive the data. The yield of the function is zero for - success, or one of the following negative numbers: - - PCRE_ERROR_NULL the argument code was NULL - the argument where was NULL - PCRE_ERROR_BADMAGIC the "magic number" was not found - PCRE_ERROR_BADENDIANNESS the pattern was compiled with different - endianness - PCRE_ERROR_BADOPTION the value of what was invalid - PCRE_ERROR_UNSET the requested field is not set - - The "magic number" is placed at the start of each compiled pattern as - an simple check against passing an arbitrary memory pointer. The endi- - anness error can occur if a compiled pattern is saved and reloaded on a - different host. Here is a typical call of pcre_fullinfo(), to obtain - the length of the compiled pattern: - - int rc; - size_t length; - rc = pcre_fullinfo( - re, /* result of pcre_compile() */ - sd, /* result of pcre_study(), or NULL */ - PCRE_INFO_SIZE, /* what is required */ - &length); /* where to put the data */ - - The possible values for the third argument are defined in pcre.h, and - are as follows: - - PCRE_INFO_BACKREFMAX - - Return the number of the highest back reference in the pattern. The - fourth argument should point to an int variable. Zero is returned if - there are no back references. - - PCRE_INFO_CAPTURECOUNT - - Return the number of capturing subpatterns in the pattern. The fourth - argument should point to an int variable. - - PCRE_INFO_DEFAULT_TABLES - - Return a pointer to the internal default character tables within PCRE. - The fourth argument should point to an unsigned char * variable. This - information call is provided for internal use by the pcre_study() func- - tion. External callers can cause PCRE to use its internal tables by - passing a NULL table pointer. - - PCRE_INFO_FIRSTBYTE (deprecated) - - Return information about the first data unit of any matched string, for - a non-anchored pattern. The name of this option refers to the 8-bit - library, where data units are bytes. The fourth argument should point - to an int variable. Negative values are used for special cases. How- - ever, this means that when the 32-bit library is in non-UTF-32 mode, - the full 32-bit range of characters cannot be returned. For this rea- - son, this value is deprecated; use PCRE_INFO_FIRSTCHARACTERFLAGS and - PCRE_INFO_FIRSTCHARACTER instead. - - If there is a fixed first value, for example, the letter "c" from a - pattern such as (cat|cow|coyote), its value is returned. In the 8-bit - library, the value is always less than 256. In the 16-bit library the - value can be up to 0xffff. In the 32-bit library the value can be up to - 0x10ffff. - - If there is no fixed first value, and if either - - (a) the pattern was compiled with the PCRE_MULTILINE option, and every - branch starts with "^", or - - (b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not - set (if it were set, the pattern would be anchored), - - -1 is returned, indicating that the pattern matches only at the start - of a subject string or after any newline within the string. Otherwise - -2 is returned. For anchored patterns, -2 is returned. - - PCRE_INFO_FIRSTCHARACTER - - Return the value of the first data unit (non-UTF character) of any - matched string in the situation where PCRE_INFO_FIRSTCHARACTERFLAGS - returns 1; otherwise return 0. The fourth argument should point to an - uint_t variable. - - In the 8-bit library, the value is always less than 256. In the 16-bit - library the value can be up to 0xffff. In the 32-bit library in UTF-32 - mode the value can be up to 0x10ffff, and up to 0xffffffff when not - using UTF-32 mode. - - PCRE_INFO_FIRSTCHARACTERFLAGS - - Return information about the first data unit of any matched string, for - a non-anchored pattern. The fourth argument should point to an int - variable. - - If there is a fixed first value, for example, the letter "c" from a - pattern such as (cat|cow|coyote), 1 is returned, and the character - value can be retrieved using PCRE_INFO_FIRSTCHARACTER. If there is no - fixed first value, and if either - - (a) the pattern was compiled with the PCRE_MULTILINE option, and every - branch starts with "^", or - - (b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not - set (if it were set, the pattern would be anchored), - - 2 is returned, indicating that the pattern matches only at the start of - a subject string or after any newline within the string. Otherwise 0 is - returned. For anchored patterns, 0 is returned. - - PCRE_INFO_FIRSTTABLE - - If the pattern was studied, and this resulted in the construction of a - 256-bit table indicating a fixed set of values for the first data unit - in any matching string, a pointer to the table is returned. Otherwise - NULL is returned. The fourth argument should point to an unsigned char - * variable. - - PCRE_INFO_HASCRORLF - - Return 1 if the pattern contains any explicit matches for CR or LF - characters, otherwise 0. The fourth argument should point to an int - variable. An explicit match is either a literal CR or LF character, or - \r or \n. - - PCRE_INFO_JCHANGED - - Return 1 if the (?J) or (?-J) option setting is used in the pattern, - otherwise 0. The fourth argument should point to an int variable. (?J) - and (?-J) set and unset the local PCRE_DUPNAMES option, respectively. - - PCRE_INFO_JIT - - Return 1 if the pattern was studied with one of the JIT options, and - just-in-time compiling was successful. The fourth argument should point - to an int variable. A return value of 0 means that JIT support is not - available in this version of PCRE, or that the pattern was not studied - with a JIT option, or that the JIT compiler could not handle this par- - ticular pattern. See the pcrejit documentation for details of what can - and cannot be handled. - - PCRE_INFO_JITSIZE - - If the pattern was successfully studied with a JIT option, return the - size of the JIT compiled code, otherwise return zero. The fourth argu- - ment should point to a size_t variable. - - PCRE_INFO_LASTLITERAL - - Return the value of the rightmost literal data unit that must exist in - any matched string, other than at its start, if such a value has been - recorded. The fourth argument should point to an int variable. If there - is no such value, -1 is returned. For anchored patterns, a last literal - value is recorded only if it follows something of variable length. For - example, for the pattern /^a\d+z\d+/ the returned value is "z", but for - /^a\dz\d/ the returned value is -1. - - Since for the 32-bit library using the non-UTF-32 mode, this function - is unable to return the full 32-bit range of characters, this value is - deprecated; instead the PCRE_INFO_REQUIREDCHARFLAGS and - PCRE_INFO_REQUIREDCHAR values should be used. - - PCRE_INFO_MATCH_EMPTY - - Return 1 if the pattern can match an empty string, otherwise 0. The - fourth argument should point to an int variable. - - PCRE_INFO_MATCHLIMIT - - If the pattern set a match limit by including an item of the form - (*LIMIT_MATCH=nnnn) at the start, the value is returned. The fourth - argument should point to an unsigned 32-bit integer. If no such value - has been set, the call to pcre_fullinfo() returns the error - PCRE_ERROR_UNSET. - - PCRE_INFO_MAXLOOKBEHIND - - Return the number of characters (NB not data units) in the longest - lookbehind assertion in the pattern. This information is useful when - doing multi-segment matching using the partial matching facilities. - Note that the simple assertions \b and \B require a one-character look- - behind. \A also registers a one-character lookbehind, though it does - not actually inspect the previous character. This is to ensure that at - least one character from the old segment is retained when a new segment - is processed. Otherwise, if there are no lookbehinds in the pattern, \A - might match incorrectly at the start of a new segment. - - PCRE_INFO_MINLENGTH - - If the pattern was studied and a minimum length for matching subject - strings was computed, its value is returned. Otherwise the returned - value is -1. The value is a number of characters, which in UTF mode may - be different from the number of data units. The fourth argument should - point to an int variable. A non-negative value is a lower bound to the - length of any matching string. There may not be any strings of that - length that do actually match, but every string that does match is at - least that long. - - PCRE_INFO_NAMECOUNT - PCRE_INFO_NAMEENTRYSIZE - PCRE_INFO_NAMETABLE - - PCRE supports the use of named as well as numbered capturing parenthe- - ses. The names are just an additional way of identifying the parenthe- - ses, which still acquire numbers. Several convenience functions such as - pcre_get_named_substring() are provided for extracting captured sub- - strings by name. It is also possible to extract the data directly, by - first converting the name to a number in order to access the correct - pointers in the output vector (described with pcre_exec() below). To do - the conversion, you need to use the name-to-number map, which is - described by these three values. - - The map consists of a number of fixed-size entries. PCRE_INFO_NAMECOUNT - gives the number of entries, and PCRE_INFO_NAMEENTRYSIZE gives the size - of each entry; both of these return an int value. The entry size - depends on the length of the longest name. PCRE_INFO_NAMETABLE returns - a pointer to the first entry of the table. This is a pointer to char in - the 8-bit library, where the first two bytes of each entry are the num- - ber of the capturing parenthesis, most significant byte first. In the - 16-bit library, the pointer points to 16-bit data units, the first of - which contains the parenthesis number. In the 32-bit library, the - pointer points to 32-bit data units, the first of which contains the - parenthesis number. The rest of the entry is the corresponding name, - zero terminated. - - The names are in alphabetical order. If (?| is used to create multiple - groups with the same number, as described in the section on duplicate - subpattern numbers in the pcrepattern page, the groups may be given the - same name, but there is only one entry in the table. Different names - for groups of the same number are not permitted. Duplicate names for - subpatterns with different numbers are permitted, but only if PCRE_DUP- - NAMES is set. They appear in the table in the order in which they were - found in the pattern. In the absence of (?| this is the order of - increasing number; when (?| is used this is not necessarily the case - because later subpatterns may have lower numbers. - - As a simple example of the name/number table, consider the following - pattern after compilation by the 8-bit library (assume PCRE_EXTENDED is - set, so white space - including newlines - is ignored): - - (?<date> (?<year>(\d\d)?\d\d) - - (?<month>\d\d) - (?<day>\d\d) ) - - There are four named subpatterns, so the table has four entries, and - each entry in the table is eight bytes long. The table is as follows, - with non-printing bytes shows in hexadecimal, and undefined bytes shown - as ??: - - 00 01 d a t e 00 ?? - 00 05 d a y 00 ?? ?? - 00 04 m o n t h 00 - 00 02 y e a r 00 ?? - - When writing code to extract data from named subpatterns using the - name-to-number map, remember that the length of the entries is likely - to be different for each compiled pattern. - - PCRE_INFO_OKPARTIAL - - Return 1 if the pattern can be used for partial matching with - pcre_exec(), otherwise 0. The fourth argument should point to an int - variable. From release 8.00, this always returns 1, because the - restrictions that previously applied to partial matching have been - lifted. The pcrepartial documentation gives details of partial match- - ing. - - PCRE_INFO_OPTIONS - - Return a copy of the options with which the pattern was compiled. The - fourth argument should point to an unsigned long int variable. These - option bits are those specified in the call to pcre_compile(), modified - by any top-level option settings at the start of the pattern itself. In - other words, they are the options that will be in force when matching - starts. For example, if the pattern /(?im)abc(?-i)d/ is compiled with - the PCRE_EXTENDED option, the result is PCRE_CASELESS, PCRE_MULTILINE, - and PCRE_EXTENDED. - - A pattern is automatically anchored by PCRE if all of its top-level - alternatives begin with one of the following: - - ^ unless PCRE_MULTILINE is set - \A always - \G always - .* if PCRE_DOTALL is set and there are no back - references to the subpattern in which .* appears - - For such patterns, the PCRE_ANCHORED bit is set in the options returned - by pcre_fullinfo(). - - PCRE_INFO_RECURSIONLIMIT - - If the pattern set a recursion limit by including an item of the form - (*LIMIT_RECURSION=nnnn) at the start, the value is returned. The fourth - argument should point to an unsigned 32-bit integer. If no such value - has been set, the call to pcre_fullinfo() returns the error - PCRE_ERROR_UNSET. - - PCRE_INFO_SIZE - - Return the size of the compiled pattern in bytes (for all three - libraries). The fourth argument should point to a size_t variable. This - value does not include the size of the pcre structure that is returned - by pcre_compile(). The value that is passed as the argument to - pcre_malloc() when pcre_compile() is getting memory in which to place - the compiled data is the value returned by this option plus the size of - the pcre structure. Studying a compiled pattern, with or without JIT, - does not alter the value returned by this option. - - PCRE_INFO_STUDYSIZE - - Return the size in bytes (for all three libraries) of the data block - pointed to by the study_data field in a pcre_extra block. If pcre_extra - is NULL, or there is no study data, zero is returned. The fourth argu- - ment should point to a size_t variable. The study_data field is set by - pcre_study() to record information that will speed up matching (see the - section entitled "Studying a pattern" above). The format of the - study_data block is private, but its length is made available via this - option so that it can be saved and restored (see the pcreprecompile - documentation for details). - - PCRE_INFO_REQUIREDCHARFLAGS - - Returns 1 if there is a rightmost literal data unit that must exist in - any matched string, other than at its start. The fourth argument should - point to an int variable. If there is no such value, 0 is returned. If - returning 1, the character value itself can be retrieved using - PCRE_INFO_REQUIREDCHAR. - - For anchored patterns, a last literal value is recorded only if it fol- - lows something of variable length. For example, for the pattern - /^a\d+z\d+/ the returned value 1 (with "z" returned from - PCRE_INFO_REQUIREDCHAR), but for /^a\dz\d/ the returned value is 0. - - PCRE_INFO_REQUIREDCHAR - - Return the value of the rightmost literal data unit that must exist in - any matched string, other than at its start, if such a value has been - recorded. The fourth argument should point to an uint32_t variable. If - there is no such value, 0 is returned. - - -REFERENCE COUNTS - - int pcre_refcount(pcre *code, int adjust); - - The pcre_refcount() function is used to maintain a reference count in - the data block that contains a compiled pattern. It is provided for the - benefit of applications that operate in an object-oriented manner, - where different parts of the application may be using the same compiled - pattern, but you want to free the block when they are all done. - - When a pattern is compiled, the reference count field is initialized to - zero. It is changed only by calling this function, whose action is to - add the adjust value (which may be positive or negative) to it. The - yield of the function is the new value. However, the value of the count - is constrained to lie between 0 and 65535, inclusive. If the new value - is outside these limits, it is forced to the appropriate limit value. - - Except when it is zero, the reference count is not correctly preserved - if a pattern is compiled on one host and then transferred to a host - whose byte-order is different. (This seems a highly unlikely scenario.) - - -MATCHING A PATTERN: THE TRADITIONAL FUNCTION - - int pcre_exec(const pcre *code, const pcre_extra *extra, - const char *subject, int length, int startoffset, - int options, int *ovector, int ovecsize); - - The function pcre_exec() is called to match a subject string against a - compiled pattern, which is passed in the code argument. If the pattern - was studied, the result of the study should be passed in the extra - argument. You can call pcre_exec() with the same code and extra argu- - ments as many times as you like, in order to match different subject - strings with the same pattern. - - This function is the main matching facility of the library, and it - operates in a Perl-like manner. For specialist use there is also an - alternative matching function, which is described below in the section - about the pcre_dfa_exec() function. - - In most applications, the pattern will have been compiled (and option- - ally studied) in the same process that calls pcre_exec(). However, it - is possible to save compiled patterns and study data, and then use them - later in different processes, possibly even on different hosts. For a - discussion about this, see the pcreprecompile documentation. - - Here is an example of a simple call to pcre_exec(): - - int rc; - int ovector[30]; - rc = pcre_exec( - re, /* result of pcre_compile() */ - NULL, /* we didn't study the pattern */ - "some string", /* the subject string */ - 11, /* the length of the subject string */ - 0, /* start at offset 0 in the subject */ - 0, /* default options */ - ovector, /* vector of integers for substring information */ - 30); /* number of elements (NOT size in bytes) */ - - Extra data for pcre_exec() - - If the extra argument is not NULL, it must point to a pcre_extra data - block. The pcre_study() function returns such a block (when it doesn't - return NULL), but you can also create one for yourself, and pass addi- - tional information in it. The pcre_extra block contains the following - fields (not necessarily in this order): - - unsigned long int flags; - void *study_data; - void *executable_jit; - unsigned long int match_limit; - unsigned long int match_limit_recursion; - void *callout_data; - const unsigned char *tables; - unsigned char **mark; - - In the 16-bit version of this structure, the mark field has type - "PCRE_UCHAR16 **". - - In the 32-bit version of this structure, the mark field has type - "PCRE_UCHAR32 **". - - The flags field is used to specify which of the other fields are set. - The flag bits are: - - PCRE_EXTRA_CALLOUT_DATA - PCRE_EXTRA_EXECUTABLE_JIT - PCRE_EXTRA_MARK - PCRE_EXTRA_MATCH_LIMIT - PCRE_EXTRA_MATCH_LIMIT_RECURSION - PCRE_EXTRA_STUDY_DATA - PCRE_EXTRA_TABLES - - Other flag bits should be set to zero. The study_data field and some- - times the executable_jit field are set in the pcre_extra block that is - returned by pcre_study(), together with the appropriate flag bits. You - should not set these yourself, but you may add to the block by setting - other fields and their corresponding flag bits. - - The match_limit field provides a means of preventing PCRE from using up - a vast amount of resources when running patterns that are not going to - match, but which have a very large number of possibilities in their - search trees. The classic example is a pattern that uses nested unlim- - ited repeats. - - Internally, pcre_exec() uses a function called match(), which it calls - repeatedly (sometimes recursively). The limit set by match_limit is - imposed on the number of times this function is called during a match, - which has the effect of limiting the amount of backtracking that can - take place. For patterns that are not anchored, the count restarts from - zero for each position in the subject string. - - When pcre_exec() is called with a pattern that was successfully studied - with a JIT option, the way that the matching is executed is entirely - different. However, there is still the possibility of runaway matching - that goes on for a very long time, and so the match_limit value is also - used in this case (but in a different way) to limit how long the match- - ing can continue. - - The default value for the limit can be set when PCRE is built; the - default default is 10 million, which handles all but the most extreme - cases. You can override the default by suppling pcre_exec() with a - pcre_extra block in which match_limit is set, and - PCRE_EXTRA_MATCH_LIMIT is set in the flags field. If the limit is - exceeded, pcre_exec() returns PCRE_ERROR_MATCHLIMIT. - - A value for the match limit may also be supplied by an item at the - start of a pattern of the form - - (*LIMIT_MATCH=d) - - where d is a decimal number. However, such a setting is ignored unless - d is less than the limit set by the caller of pcre_exec() or, if no - such limit is set, less than the default. - - The match_limit_recursion field is similar to match_limit, but instead - of limiting the total number of times that match() is called, it limits - the depth of recursion. The recursion depth is a smaller number than - the total number of calls, because not all calls to match() are recur- - sive. This limit is of use only if it is set smaller than match_limit. - - Limiting the recursion depth limits the amount of machine stack that - can be used, or, when PCRE has been compiled to use memory on the heap - instead of the stack, the amount of heap memory that can be used. This - limit is not relevant, and is ignored, when matching is done using JIT - compiled code. - - The default value for match_limit_recursion can be set when PCRE is - built; the default default is the same value as the default for - match_limit. You can override the default by suppling pcre_exec() with - a pcre_extra block in which match_limit_recursion is set, and - PCRE_EXTRA_MATCH_LIMIT_RECURSION is set in the flags field. If the - limit is exceeded, pcre_exec() returns PCRE_ERROR_RECURSIONLIMIT. - - A value for the recursion limit may also be supplied by an item at the - start of a pattern of the form - - (*LIMIT_RECURSION=d) - - where d is a decimal number. However, such a setting is ignored unless - d is less than the limit set by the caller of pcre_exec() or, if no - such limit is set, less than the default. - - The callout_data field is used in conjunction with the "callout" fea- - ture, and is described in the pcrecallout documentation. - - The tables field is provided for use with patterns that have been pre- - compiled using custom character tables, saved to disc or elsewhere, and - then reloaded, because the tables that were used to compile a pattern - are not saved with it. See the pcreprecompile documentation for a dis- - cussion of saving compiled patterns for later use. If NULL is passed - using this mechanism, it forces PCRE's internal tables to be used. - - Warning: The tables that pcre_exec() uses must be the same as those - that were used when the pattern was compiled. If this is not the case, - the behaviour of pcre_exec() is undefined. Therefore, when a pattern is - compiled and matched in the same process, this field should never be - set. In this (the most common) case, the correct table pointer is auto- - matically passed with the compiled pattern from pcre_compile() to - pcre_exec(). - - If PCRE_EXTRA_MARK is set in the flags field, the mark field must be - set to point to a suitable variable. If the pattern contains any back- - tracking control verbs such as (*MARK:NAME), and the execution ends up - with a name to pass back, a pointer to the name string (zero termi- - nated) is placed in the variable pointed to by the mark field. The - names are within the compiled pattern; if you wish to retain such a - name you must copy it before freeing the memory of a compiled pattern. - If there is no name to pass back, the variable pointed to by the mark - field is set to NULL. For details of the backtracking control verbs, - see the section entitled "Backtracking control" in the pcrepattern doc- - umentation. - - Option bits for pcre_exec() - - The unused bits of the options argument for pcre_exec() must be zero. - The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_xxx, - PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, - PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_HARD, and - PCRE_PARTIAL_SOFT. - - If the pattern was successfully studied with one of the just-in-time - (JIT) compile options, the only supported options for JIT execution are - PCRE_NO_UTF8_CHECK, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, - PCRE_NOTEMPTY_ATSTART, PCRE_PARTIAL_HARD, and PCRE_PARTIAL_SOFT. If an - unsupported option is used, JIT execution is disabled and the normal - interpretive code in pcre_exec() is run. - - PCRE_ANCHORED - - The PCRE_ANCHORED option limits pcre_exec() to matching at the first - matching position. If a pattern was compiled with PCRE_ANCHORED, or - turned out to be anchored by virtue of its contents, it cannot be made - unachored at matching time. - - PCRE_BSR_ANYCRLF - PCRE_BSR_UNICODE - - These options (which are mutually exclusive) control what the \R escape - sequence matches. The choice is either to match only CR, LF, or CRLF, - or to match any Unicode newline sequence. These options override the - choice that was made or defaulted when the pattern was compiled. - - PCRE_NEWLINE_CR - PCRE_NEWLINE_LF - PCRE_NEWLINE_CRLF - PCRE_NEWLINE_ANYCRLF - PCRE_NEWLINE_ANY - - These options override the newline definition that was chosen or - defaulted when the pattern was compiled. For details, see the descrip- - tion of pcre_compile() above. During matching, the newline choice - affects the behaviour of the dot, circumflex, and dollar metacharac- - ters. It may also alter the way the match position is advanced after a - match failure for an unanchored pattern. - - When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is - set, and a match attempt for an unanchored pattern fails when the cur- - rent position is at a CRLF sequence, and the pattern contains no - explicit matches for CR or LF characters, the match position is - advanced by two characters instead of one, in other words, to after the - CRLF. - - The above rule is a compromise that makes the most common cases work as - expected. For example, if the pattern is .+A (and the PCRE_DOTALL - option is not set), it does not match the string "\r\nA" because, after - failing at the start, it skips both the CR and the LF before retrying. - However, the pattern [\r\n]A does match that string, because it con- - tains an explicit CR or LF reference, and so advances only by one char- - acter after the first failure. - - An explicit match for CR of LF is either a literal appearance of one of - those characters, or one of the \r or \n escape sequences. Implicit - matches such as [^X] do not count, nor does \s (which includes CR and - LF in the characters that it matches). - - Notwithstanding the above, anomalous effects may still occur when CRLF - is a valid newline sequence and explicit \r or \n escapes appear in the - pattern. - - PCRE_NOTBOL - - This option specifies that first character of the subject string is not - the beginning of a line, so the circumflex metacharacter should not - match before it. Setting this without PCRE_MULTILINE (at compile time) - causes circumflex never to match. This option affects only the behav- - iour of the circumflex metacharacter. It does not affect \A. - - PCRE_NOTEOL - - This option specifies that the end of the subject string is not the end - of a line, so the dollar metacharacter should not match it nor (except - in multiline mode) a newline immediately before it. Setting this with- - out PCRE_MULTILINE (at compile time) causes dollar never to match. This - option affects only the behaviour of the dollar metacharacter. It does - not affect \Z or \z. - - PCRE_NOTEMPTY - - An empty string is not considered to be a valid match if this option is - set. If there are alternatives in the pattern, they are tried. If all - the alternatives match the empty string, the entire match fails. For - example, if the pattern - - a?b? - - is applied to a string not beginning with "a" or "b", it matches an - empty string at the start of the subject. With PCRE_NOTEMPTY set, this - match is not valid, so PCRE searches further into the string for occur- - rences of "a" or "b". - - PCRE_NOTEMPTY_ATSTART - - This is like PCRE_NOTEMPTY, except that an empty string match that is - not at the start of the subject is permitted. If the pattern is - anchored, such a match can occur only if the pattern contains \K. - - Perl has no direct equivalent of PCRE_NOTEMPTY or - PCRE_NOTEMPTY_ATSTART, but it does make a special case of a pattern - match of the empty string within its split() function, and when using - the /g modifier. It is possible to emulate Perl's behaviour after - matching a null string by first trying the match again at the same off- - set with PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED, and then if that - fails, by advancing the starting offset (see below) and trying an ordi- - nary match again. There is some code that demonstrates how to do this - in the pcredemo sample program. In the most general case, you have to - check to see if the newline convention recognizes CRLF as a newline, - and if so, and the current character is CR followed by LF, advance the - starting offset by two characters instead of one. - - PCRE_NO_START_OPTIMIZE - - There are a number of optimizations that pcre_exec() uses at the start - of a match, in order to speed up the process. For example, if it is - known that an unanchored match must start with a specific character, it - searches the subject for that character, and fails immediately if it - cannot find it, without actually running the main matching function. - This means that a special item such as (*COMMIT) at the start of a pat- - tern is not considered until after a suitable starting point for the - match has been found. Also, when callouts or (*MARK) items are in use, - these "start-up" optimizations can cause them to be skipped if the pat- - tern is never actually used. The start-up optimizations are in effect a - pre-scan of the subject that takes place before the pattern is run. - - The PCRE_NO_START_OPTIMIZE option disables the start-up optimizations, - possibly causing performance to suffer, but ensuring that in cases - where the result is "no match", the callouts do occur, and that items - such as (*COMMIT) and (*MARK) are considered at every possible starting - position in the subject string. If PCRE_NO_START_OPTIMIZE is set at - compile time, it cannot be unset at matching time. The use of - PCRE_NO_START_OPTIMIZE at matching time (that is, passing it to - pcre_exec()) disables JIT execution; in this situation, matching is - always done using interpretively. - - Setting PCRE_NO_START_OPTIMIZE can change the outcome of a matching - operation. Consider the pattern - - (*COMMIT)ABC - - When this is compiled, PCRE records the fact that a match must start - with the character "A". Suppose the subject string is "DEFABC". The - start-up optimization scans along the subject, finds "A" and runs the - first match attempt from there. The (*COMMIT) item means that the pat- - tern must match the current starting position, which in this case, it - does. However, if the same match is run with PCRE_NO_START_OPTIMIZE - set, the initial scan along the subject string does not happen. The - first match attempt is run starting from "D" and when this fails, - (*COMMIT) prevents any further matches being tried, so the overall - result is "no match". If the pattern is studied, more start-up opti- - mizations may be used. For example, a minimum length for the subject - may be recorded. Consider the pattern - - (*MARK:A)(X|Y) - - The minimum length for a match is one character. If the subject is - "ABC", there will be attempts to match "ABC", "BC", "C", and then - finally an empty string. If the pattern is studied, the final attempt - does not take place, because PCRE knows that the subject is too short, - and so the (*MARK) is never encountered. In this case, studying the - pattern does not affect the overall match result, which is still "no - match", but it does affect the auxiliary information that is returned. - - PCRE_NO_UTF8_CHECK - - When PCRE_UTF8 is set at compile time, the validity of the subject as a - UTF-8 string is automatically checked when pcre_exec() is subsequently - called. The entire string is checked before any other processing takes - place. The value of startoffset is also checked to ensure that it - points to the start of a UTF-8 character. There is a discussion about - the validity of UTF-8 strings in the pcreunicode page. If an invalid - sequence of bytes is found, pcre_exec() returns the error - PCRE_ERROR_BADUTF8 or, if PCRE_PARTIAL_HARD is set and the problem is a - truncated character at the end of the subject, PCRE_ERROR_SHORTUTF8. In - both cases, information about the precise nature of the error may also - be returned (see the descriptions of these errors in the section enti- - tled Error return values from pcre_exec() below). If startoffset con- - tains a value that does not point to the start of a UTF-8 character (or - to the end of the subject), PCRE_ERROR_BADUTF8_OFFSET is returned. - - If you already know that your subject is valid, and you want to skip - these checks for performance reasons, you can set the - PCRE_NO_UTF8_CHECK option when calling pcre_exec(). You might want to - do this for the second and subsequent calls to pcre_exec() if you are - making repeated calls to find all the matches in a single subject - string. However, you should be sure that the value of startoffset - points to the start of a character (or the end of the subject). When - PCRE_NO_UTF8_CHECK is set, the effect of passing an invalid string as a - subject or an invalid value of startoffset is undefined. Your program - may crash or loop. - - PCRE_PARTIAL_HARD - PCRE_PARTIAL_SOFT - - These options turn on the partial matching feature. For backwards com- - patibility, PCRE_PARTIAL is a synonym for PCRE_PARTIAL_SOFT. A partial - match occurs if the end of the subject string is reached successfully, - but there are not enough subject characters to complete the match. If - this happens when PCRE_PARTIAL_SOFT (but not PCRE_PARTIAL_HARD) is set, - matching continues by testing any remaining alternatives. Only if no - complete match can be found is PCRE_ERROR_PARTIAL returned instead of - PCRE_ERROR_NOMATCH. In other words, PCRE_PARTIAL_SOFT says that the - caller is prepared to handle a partial match, but only if no complete - match can be found. - - If PCRE_PARTIAL_HARD is set, it overrides PCRE_PARTIAL_SOFT. In this - case, if a partial match is found, pcre_exec() immediately returns - PCRE_ERROR_PARTIAL, without considering any other alternatives. In - other words, when PCRE_PARTIAL_HARD is set, a partial match is consid- - ered to be more important that an alternative complete match. - - In both cases, the portion of the string that was inspected when the - partial match was found is set as the first matching string. There is a - more detailed discussion of partial and multi-segment matching, with - examples, in the pcrepartial documentation. - - The string to be matched by pcre_exec() - - The subject string is passed to pcre_exec() as a pointer in subject, a - length in length, and a starting offset in startoffset. The units for - length and startoffset are bytes for the 8-bit library, 16-bit data - items for the 16-bit library, and 32-bit data items for the 32-bit - library. - - If startoffset is negative or greater than the length of the subject, - pcre_exec() returns PCRE_ERROR_BADOFFSET. When the starting offset is - zero, the search for a match starts at the beginning of the subject, - and this is by far the most common case. In UTF-8 or UTF-16 mode, the - offset must point to the start of a character, or the end of the sub- - ject (in UTF-32 mode, one data unit equals one character, so all off- - sets are valid). Unlike the pattern string, the subject may contain - binary zeroes. - - A non-zero starting offset is useful when searching for another match - in the same subject by calling pcre_exec() again after a previous suc- - cess. Setting startoffset differs from just passing over a shortened - string and setting PCRE_NOTBOL in the case of a pattern that begins - with any kind of lookbehind. For example, consider the pattern - - \Biss\B - - which finds occurrences of "iss" in the middle of words. (\B matches - only if the current position in the subject is not a word boundary.) - When applied to the string "Mississipi" the first call to pcre_exec() - finds the first occurrence. If pcre_exec() is called again with just - the remainder of the subject, namely "issipi", it does not match, - because \B is always false at the start of the subject, which is deemed - to be a word boundary. However, if pcre_exec() is passed the entire - string again, but with startoffset set to 4, it finds the second occur- - rence of "iss" because it is able to look behind the starting point to - discover that it is preceded by a letter. - - Finding all the matches in a subject is tricky when the pattern can - match an empty string. It is possible to emulate Perl's /g behaviour by - first trying the match again at the same offset, with the - PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED options, and then if that - fails, advancing the starting offset and trying an ordinary match - again. There is some code that demonstrates how to do this in the pcre- - demo sample program. In the most general case, you have to check to see - if the newline convention recognizes CRLF as a newline, and if so, and - the current character is CR followed by LF, advance the starting offset - by two characters instead of one. - - If a non-zero starting offset is passed when the pattern is anchored, - one attempt to match at the given offset is made. This can only succeed - if the pattern does not require the match to be at the start of the - subject. - - How pcre_exec() returns captured substrings - - In general, a pattern matches a certain portion of the subject, and in - addition, further substrings from the subject may be picked out by - parts of the pattern. Following the usage in Jeffrey Friedl's book, - this is called "capturing" in what follows, and the phrase "capturing - subpattern" is used for a fragment of a pattern that picks out a sub- - string. PCRE supports several other kinds of parenthesized subpattern - that do not cause substrings to be captured. - - Captured substrings are returned to the caller via a vector of integers - whose address is passed in ovector. The number of elements in the vec- - tor is passed in ovecsize, which must be a non-negative number. Note: - this argument is NOT the size of ovector in bytes. - - The first two-thirds of the vector is used to pass back captured sub- - strings, each substring using a pair of integers. The remaining third - of the vector is used as workspace by pcre_exec() while matching cap- - turing subpatterns, and is not available for passing back information. - The number passed in ovecsize should always be a multiple of three. If - it is not, it is rounded down. - - When a match is successful, information about captured substrings is - returned in pairs of integers, starting at the beginning of ovector, - and continuing up to two-thirds of its length at the most. The first - element of each pair is set to the offset of the first character in a - substring, and the second is set to the offset of the first character - after the end of a substring. These values are always data unit off- - sets, even in UTF mode. They are byte offsets in the 8-bit library, - 16-bit data item offsets in the 16-bit library, and 32-bit data item - offsets in the 32-bit library. Note: they are not character counts. - - The first pair of integers, ovector[0] and ovector[1], identify the - portion of the subject string matched by the entire pattern. The next - pair is used for the first capturing subpattern, and so on. The value - returned by pcre_exec() is one more than the highest numbered pair that - has been set. For example, if two substrings have been captured, the - returned value is 3. If there are no capturing subpatterns, the return - value from a successful match is 1, indicating that just the first pair - of offsets has been set. - - If a capturing subpattern is matched repeatedly, it is the last portion - of the string that it matched that is returned. - - If the vector is too small to hold all the captured substring offsets, - it is used as far as possible (up to two-thirds of its length), and the - function returns a value of zero. If neither the actual string matched - nor any captured substrings are of interest, pcre_exec() may be called - with ovector passed as NULL and ovecsize as zero. However, if the pat- - tern contains back references and the ovector is not big enough to - remember the related substrings, PCRE has to get additional memory for - use during matching. Thus it is usually advisable to supply an ovector - of reasonable size. - - There are some cases where zero is returned (indicating vector over- - flow) when in fact the vector is exactly the right size for the final - match. For example, consider the pattern - - (a)(?:(b)c|bd) - - If a vector of 6 elements (allowing for only 1 captured substring) is - given with subject string "abd", pcre_exec() will try to set the second - captured string, thereby recording a vector overflow, before failing to - match "c" and backing up to try the second alternative. The zero - return, however, does correctly indicate that the maximum number of - slots (namely 2) have been filled. In similar cases where there is tem- - porary overflow, but the final number of used slots is actually less - than the maximum, a non-zero value is returned. - - The pcre_fullinfo() function can be used to find out how many capturing - subpatterns there are in a compiled pattern. The smallest size for - ovector that will allow for n captured substrings, in addition to the - offsets of the substring matched by the whole pattern, is (n+1)*3. - - It is possible for capturing subpattern number n+1 to match some part - of the subject when subpattern n has not been used at all. For example, - if the string "abc" is matched against the pattern (a|(z))(bc) the - return from the function is 4, and subpatterns 1 and 3 are matched, but - 2 is not. When this happens, both values in the offset pairs corre- - sponding to unused subpatterns are set to -1. - - Offset values that correspond to unused subpatterns at the end of the - expression are also set to -1. For example, if the string "abc" is - matched against the pattern (abc)(x(yz)?)? subpatterns 2 and 3 are not - matched. The return from the function is 2, because the highest used - capturing subpattern number is 1, and the offsets for for the second - and third capturing subpatterns (assuming the vector is large enough, - of course) are set to -1. - - Note: Elements in the first two-thirds of ovector that do not corre- - spond to capturing parentheses in the pattern are never changed. That - is, if a pattern contains n capturing parentheses, no more than ovec- - tor[0] to ovector[2n+1] are set by pcre_exec(). The other elements (in - the first two-thirds) retain whatever values they previously had. - - Some convenience functions are provided for extracting the captured - substrings as separate strings. These are described below. - - Error return values from pcre_exec() - - If pcre_exec() fails, it returns a negative number. The following are - defined in the header file: - - PCRE_ERROR_NOMATCH (-1) - - The subject string did not match the pattern. - - PCRE_ERROR_NULL (-2) - - Either code or subject was passed as NULL, or ovector was NULL and - ovecsize was not zero. - - PCRE_ERROR_BADOPTION (-3) - - An unrecognized bit was set in the options argument. - - PCRE_ERROR_BADMAGIC (-4) - - PCRE stores a 4-byte "magic number" at the start of the compiled code, - to catch the case when it is passed a junk pointer and to detect when a - pattern that was compiled in an environment of one endianness is run in - an environment with the other endianness. This is the error that PCRE - gives when the magic number is not present. - - PCRE_ERROR_UNKNOWN_OPCODE (-5) - - While running the pattern match, an unknown item was encountered in the - compiled pattern. This error could be caused by a bug in PCRE or by - overwriting of the compiled pattern. - - PCRE_ERROR_NOMEMORY (-6) - - If a pattern contains back references, but the ovector that is passed - to pcre_exec() is not big enough to remember the referenced substrings, - PCRE gets a block of memory at the start of matching to use for this - purpose. If the call via pcre_malloc() fails, this error is given. The - memory is automatically freed at the end of matching. - - This error is also given if pcre_stack_malloc() fails in pcre_exec(). - This can happen only when PCRE has been compiled with --disable-stack- - for-recursion. - - PCRE_ERROR_NOSUBSTRING (-7) - - This error is used by the pcre_copy_substring(), pcre_get_substring(), - and pcre_get_substring_list() functions (see below). It is never - returned by pcre_exec(). - - PCRE_ERROR_MATCHLIMIT (-8) - - The backtracking limit, as specified by the match_limit field in a - pcre_extra structure (or defaulted) was reached. See the description - above. - - PCRE_ERROR_CALLOUT (-9) - - This error is never generated by pcre_exec() itself. It is provided for - use by callout functions that want to yield a distinctive error code. - See the pcrecallout documentation for details. - - PCRE_ERROR_BADUTF8 (-10) - - A string that contains an invalid UTF-8 byte sequence was passed as a - subject, and the PCRE_NO_UTF8_CHECK option was not set. If the size of - the output vector (ovecsize) is at least 2, the byte offset to the - start of the the invalid UTF-8 character is placed in the first ele- - ment, and a reason code is placed in the second element. The reason - codes are listed in the following section. For backward compatibility, - if PCRE_PARTIAL_HARD is set and the problem is a truncated UTF-8 char- - acter at the end of the subject (reason codes 1 to 5), - PCRE_ERROR_SHORTUTF8 is returned instead of PCRE_ERROR_BADUTF8. - - PCRE_ERROR_BADUTF8_OFFSET (-11) - - The UTF-8 byte sequence that was passed as a subject was checked and - found to be valid (the PCRE_NO_UTF8_CHECK option was not set), but the - value of startoffset did not point to the beginning of a UTF-8 charac- - ter or the end of the subject. - - PCRE_ERROR_PARTIAL (-12) - - The subject string did not match, but it did match partially. See the - pcrepartial documentation for details of partial matching. - - PCRE_ERROR_BADPARTIAL (-13) - - This code is no longer in use. It was formerly returned when the - PCRE_PARTIAL option was used with a compiled pattern containing items - that were not supported for partial matching. From release 8.00 - onwards, there are no restrictions on partial matching. - - PCRE_ERROR_INTERNAL (-14) - - An unexpected internal error has occurred. This error could be caused - by a bug in PCRE or by overwriting of the compiled pattern. - - PCRE_ERROR_BADCOUNT (-15) - - This error is given if the value of the ovecsize argument is negative. - - PCRE_ERROR_RECURSIONLIMIT (-21) - - The internal recursion limit, as specified by the match_limit_recursion - field in a pcre_extra structure (or defaulted) was reached. See the - description above. - - PCRE_ERROR_BADNEWLINE (-23) - - An invalid combination of PCRE_NEWLINE_xxx options was given. - - PCRE_ERROR_BADOFFSET (-24) - - The value of startoffset was negative or greater than the length of the - subject, that is, the value in length. - - PCRE_ERROR_SHORTUTF8 (-25) - - This error is returned instead of PCRE_ERROR_BADUTF8 when the subject - string ends with a truncated UTF-8 character and the PCRE_PARTIAL_HARD - option is set. Information about the failure is returned as for - PCRE_ERROR_BADUTF8. It is in fact sufficient to detect this case, but - this special error code for PCRE_PARTIAL_HARD precedes the implementa- - tion of returned information; it is retained for backwards compatibil- - ity. - - PCRE_ERROR_RECURSELOOP (-26) - - This error is returned when pcre_exec() detects a recursion loop within - the pattern. Specifically, it means that either the whole pattern or a - subpattern has been called recursively for the second time at the same - position in the subject string. Some simple patterns that might do this - are detected and faulted at compile time, but more complicated cases, - in particular mutual recursions between two different subpatterns, can- - not be detected until run time. - - PCRE_ERROR_JIT_STACKLIMIT (-27) - - This error is returned when a pattern that was successfully studied - using a JIT compile option is being matched, but the memory available - for the just-in-time processing stack is not large enough. See the - pcrejit documentation for more details. - - PCRE_ERROR_BADMODE (-28) - - This error is given if a pattern that was compiled by the 8-bit library - is passed to a 16-bit or 32-bit library function, or vice versa. - - PCRE_ERROR_BADENDIANNESS (-29) - - This error is given if a pattern that was compiled and saved is - reloaded on a host with different endianness. The utility function - pcre_pattern_to_host_byte_order() can be used to convert such a pattern - so that it runs on the new host. - - PCRE_ERROR_JIT_BADOPTION - - This error is returned when a pattern that was successfully studied - using a JIT compile option is being matched, but the matching mode - (partial or complete match) does not correspond to any JIT compilation - mode. When the JIT fast path function is used, this error may be also - given for invalid options. See the pcrejit documentation for more - details. - - PCRE_ERROR_BADLENGTH (-32) - - This error is given if pcre_exec() is called with a negative value for - the length argument. - - Error numbers -16 to -20, -22, and 30 are not used by pcre_exec(). - - Reason codes for invalid UTF-8 strings - - This section applies only to the 8-bit library. The corresponding - information for the 16-bit and 32-bit libraries is given in the pcre16 - and pcre32 pages. - - When pcre_exec() returns either PCRE_ERROR_BADUTF8 or PCRE_ERROR_SHORT- - UTF8, and the size of the output vector (ovecsize) is at least 2, the - offset of the start of the invalid UTF-8 character is placed in the - first output vector element (ovector[0]) and a reason code is placed in - the second element (ovector[1]). The reason codes are given names in - the pcre.h header file: - - PCRE_UTF8_ERR1 - PCRE_UTF8_ERR2 - PCRE_UTF8_ERR3 - PCRE_UTF8_ERR4 - PCRE_UTF8_ERR5 - - The string ends with a truncated UTF-8 character; the code specifies - how many bytes are missing (1 to 5). Although RFC 3629 restricts UTF-8 - characters to be no longer than 4 bytes, the encoding scheme (origi- - nally defined by RFC 2279) allows for up to 6 bytes, and this is - checked first; hence the possibility of 4 or 5 missing bytes. - - PCRE_UTF8_ERR6 - PCRE_UTF8_ERR7 - PCRE_UTF8_ERR8 - PCRE_UTF8_ERR9 - PCRE_UTF8_ERR10 - - The two most significant bits of the 2nd, 3rd, 4th, 5th, or 6th byte of - the character do not have the binary value 0b10 (that is, either the - most significant bit is 0, or the next bit is 1). - - PCRE_UTF8_ERR11 - PCRE_UTF8_ERR12 - - A character that is valid by the RFC 2279 rules is either 5 or 6 bytes - long; these code points are excluded by RFC 3629. - - PCRE_UTF8_ERR13 - - A 4-byte character has a value greater than 0x10fff; these code points - are excluded by RFC 3629. - - PCRE_UTF8_ERR14 - - A 3-byte character has a value in the range 0xd800 to 0xdfff; this - range of code points are reserved by RFC 3629 for use with UTF-16, and - so are excluded from UTF-8. - - PCRE_UTF8_ERR15 - PCRE_UTF8_ERR16 - PCRE_UTF8_ERR17 - PCRE_UTF8_ERR18 - PCRE_UTF8_ERR19 - - A 2-, 3-, 4-, 5-, or 6-byte character is "overlong", that is, it codes - for a value that can be represented by fewer bytes, which is invalid. - For example, the two bytes 0xc0, 0xae give the value 0x2e, whose cor- - rect coding uses just one byte. - - PCRE_UTF8_ERR20 - - The two most significant bits of the first byte of a character have the - binary value 0b10 (that is, the most significant bit is 1 and the sec- - ond is 0). Such a byte can only validly occur as the second or subse- - quent byte of a multi-byte character. - - PCRE_UTF8_ERR21 - - The first byte of a character has the value 0xfe or 0xff. These values - can never occur in a valid UTF-8 string. - - PCRE_UTF8_ERR22 - - This error code was formerly used when the presence of a so-called - "non-character" caused an error. Unicode corrigendum #9 makes it clear - that such characters should not cause a string to be rejected, and so - this code is no longer in use and is never returned. - - -EXTRACTING CAPTURED SUBSTRINGS BY NUMBER - - int pcre_copy_substring(const char *subject, int *ovector, - int stringcount, int stringnumber, char *buffer, - int buffersize); - - int pcre_get_substring(const char *subject, int *ovector, - int stringcount, int stringnumber, - const char **stringptr); - - int pcre_get_substring_list(const char *subject, - int *ovector, int stringcount, const char ***listptr); - - Captured substrings can be accessed directly by using the offsets - returned by pcre_exec() in ovector. For convenience, the functions - pcre_copy_substring(), pcre_get_substring(), and pcre_get_sub- - string_list() are provided for extracting captured substrings as new, - separate, zero-terminated strings. These functions identify substrings - by number. The next section describes functions for extracting named - substrings. - - A substring that contains a binary zero is correctly extracted and has - a further zero added on the end, but the result is not, of course, a C - string. However, you can process such a string by referring to the - length that is returned by pcre_copy_substring() and pcre_get_sub- - string(). Unfortunately, the interface to pcre_get_substring_list() is - not adequate for handling strings containing binary zeros, because the - end of the final string is not independently indicated. - - The first three arguments are the same for all three of these func- - tions: subject is the subject string that has just been successfully - matched, ovector is a pointer to the vector of integer offsets that was - passed to pcre_exec(), and stringcount is the number of substrings that - were captured by the match, including the substring that matched the - entire regular expression. This is the value returned by pcre_exec() if - it is greater than zero. If pcre_exec() returned zero, indicating that - it ran out of space in ovector, the value passed as stringcount should - be the number of elements in the vector divided by three. - - The functions pcre_copy_substring() and pcre_get_substring() extract a - single substring, whose number is given as stringnumber. A value of - zero extracts the substring that matched the entire pattern, whereas - higher values extract the captured substrings. For pcre_copy_sub- - string(), the string is placed in buffer, whose length is given by - buffersize, while for pcre_get_substring() a new block of memory is - obtained via pcre_malloc, and its address is returned via stringptr. - The yield of the function is the length of the string, not including - the terminating zero, or one of these error codes: - - PCRE_ERROR_NOMEMORY (-6) - - The buffer was too small for pcre_copy_substring(), or the attempt to - get memory failed for pcre_get_substring(). - - PCRE_ERROR_NOSUBSTRING (-7) - - There is no substring whose number is stringnumber. - - The pcre_get_substring_list() function extracts all available sub- - strings and builds a list of pointers to them. All this is done in a - single block of memory that is obtained via pcre_malloc. The address of - the memory block is returned via listptr, which is also the start of - the list of string pointers. The end of the list is marked by a NULL - pointer. The yield of the function is zero if all went well, or the - error code - - PCRE_ERROR_NOMEMORY (-6) - - if the attempt to get the memory block failed. - - When any of these functions encounter a substring that is unset, which - can happen when capturing subpattern number n+1 matches some part of - the subject, but subpattern n has not been used at all, they return an - empty string. This can be distinguished from a genuine zero-length sub- - string by inspecting the appropriate offset in ovector, which is nega- - tive for unset substrings. - - The two convenience functions pcre_free_substring() and pcre_free_sub- - string_list() can be used to free the memory returned by a previous - call of pcre_get_substring() or pcre_get_substring_list(), respec- - tively. They do nothing more than call the function pointed to by - pcre_free, which of course could be called directly from a C program. - However, PCRE is used in some situations where it is linked via a spe- - cial interface to another programming language that cannot use - pcre_free directly; it is for these cases that the functions are pro- - vided. - - -EXTRACTING CAPTURED SUBSTRINGS BY NAME - - int pcre_get_stringnumber(const pcre *code, - const char *name); - - int pcre_copy_named_substring(const pcre *code, - const char *subject, int *ovector, - int stringcount, const char *stringname, - char *buffer, int buffersize); - - int pcre_get_named_substring(const pcre *code, - const char *subject, int *ovector, - int stringcount, const char *stringname, - const char **stringptr); - - To extract a substring by name, you first have to find associated num- - ber. For example, for this pattern - - (a+)b(?<xxx>\d+)... - - the number of the subpattern called "xxx" is 2. If the name is known to - be unique (PCRE_DUPNAMES was not set), you can find the number from the - name by calling pcre_get_stringnumber(). The first argument is the com- - piled pattern, and the second is the name. The yield of the function is - the subpattern number, or PCRE_ERROR_NOSUBSTRING (-7) if there is no - subpattern of that name. - - Given the number, you can extract the substring directly, or use one of - the functions described in the previous section. For convenience, there - are also two functions that do the whole job. - - Most of the arguments of pcre_copy_named_substring() and - pcre_get_named_substring() are the same as those for the similarly - named functions that extract by number. As these are described in the - previous section, they are not re-described here. There are just two - differences: - - First, instead of a substring number, a substring name is given. Sec- - ond, there is an extra argument, given at the start, which is a pointer - to the compiled pattern. This is needed in order to gain access to the - name-to-number translation table. - - These functions call pcre_get_stringnumber(), and if it succeeds, they - then call pcre_copy_substring() or pcre_get_substring(), as appropri- - ate. NOTE: If PCRE_DUPNAMES is set and there are duplicate names, the - behaviour may not be what you want (see the next section). - - Warning: If the pattern uses the (?| feature to set up multiple subpat- - terns with the same number, as described in the section on duplicate - subpattern numbers in the pcrepattern page, you cannot use names to - distinguish the different subpatterns, because names are not included - in the compiled code. The matching process uses only numbers. For this - reason, the use of different names for subpatterns of the same number - causes an error at compile time. - - -DUPLICATE SUBPATTERN NAMES - - int pcre_get_stringtable_entries(const pcre *code, - const char *name, char **first, char **last); - - When a pattern is compiled with the PCRE_DUPNAMES option, names for - subpatterns are not required to be unique. (Duplicate names are always - allowed for subpatterns with the same number, created by using the (?| - feature. Indeed, if such subpatterns are named, they are required to - use the same names.) - - Normally, patterns with duplicate names are such that in any one match, - only one of the named subpatterns participates. An example is shown in - the pcrepattern documentation. - - When duplicates are present, pcre_copy_named_substring() and - pcre_get_named_substring() return the first substring corresponding to - the given name that is set. If none are set, PCRE_ERROR_NOSUBSTRING - (-7) is returned; no data is returned. The pcre_get_stringnumber() - function returns one of the numbers that are associated with the name, - but it is not defined which it is. - - If you want to get full details of all captured substrings for a given - name, you must use the pcre_get_stringtable_entries() function. The - first argument is the compiled pattern, and the second is the name. The - third and fourth are pointers to variables which are updated by the - function. After it has run, they point to the first and last entries in - the name-to-number table for the given name. The function itself - returns the length of each entry, or PCRE_ERROR_NOSUBSTRING (-7) if - there are none. The format of the table is described above in the sec- - tion entitled Information about a pattern above. Given all the rele- - vant entries for the name, you can extract each of their numbers, and - hence the captured data, if any. - - -FINDING ALL POSSIBLE MATCHES - - The traditional matching function uses a similar algorithm to Perl, - which stops when it finds the first match, starting at a given point in - the subject. If you want to find all possible matches, or the longest - possible match, consider using the alternative matching function (see - below) instead. If you cannot use the alternative function, but still - need to find all possible matches, you can kludge it up by making use - of the callout facility, which is described in the pcrecallout documen- - tation. - - What you have to do is to insert a callout right at the end of the pat- - tern. When your callout function is called, extract and save the cur- - rent matched substring. Then return 1, which forces pcre_exec() to - backtrack and try other alternatives. Ultimately, when it runs out of - matches, pcre_exec() will yield PCRE_ERROR_NOMATCH. - - -OBTAINING AN ESTIMATE OF STACK USAGE - - Matching certain patterns using pcre_exec() can use a lot of process - stack, which in certain environments can be rather limited in size. - Some users find it helpful to have an estimate of the amount of stack - that is used by pcre_exec(), to help them set recursion limits, as - described in the pcrestack documentation. The estimate that is output - by pcretest when called with the -m and -C options is obtained by call- - ing pcre_exec with the values NULL, NULL, NULL, -999, and -999 for its - first five arguments. - - Normally, if its first argument is NULL, pcre_exec() immediately - returns the negative error code PCRE_ERROR_NULL, but with this special - combination of arguments, it returns instead a negative number whose - absolute value is the approximate stack frame size in bytes. (A nega- - tive number is used so that it is clear that no match has happened.) - The value is approximate because in some cases, recursive calls to - pcre_exec() occur when there are one or two additional variables on the - stack. - - If PCRE has been compiled to use the heap instead of the stack for - recursion, the value returned is the size of each block that is - obtained from the heap. - - -MATCHING A PATTERN: THE ALTERNATIVE FUNCTION - - int pcre_dfa_exec(const pcre *code, const pcre_extra *extra, - const char *subject, int length, int startoffset, - int options, int *ovector, int ovecsize, - int *workspace, int wscount); - - The function pcre_dfa_exec() is called to match a subject string - against a compiled pattern, using a matching algorithm that scans the - subject string just once, and does not backtrack. This has different - characteristics to the normal algorithm, and is not compatible with - Perl. Some of the features of PCRE patterns are not supported. Never- - theless, there are times when this kind of matching can be useful. For - a discussion of the two matching algorithms, and a list of features - that pcre_dfa_exec() does not support, see the pcrematching documenta- - tion. - - The arguments for the pcre_dfa_exec() function are the same as for - pcre_exec(), plus two extras. The ovector argument is used in a differ- - ent way, and this is described below. The other common arguments are - used in the same way as for pcre_exec(), so their description is not - repeated here. - - The two additional arguments provide workspace for the function. The - workspace vector should contain at least 20 elements. It is used for - keeping track of multiple paths through the pattern tree. More - workspace will be needed for patterns and subjects where there are a - lot of potential matches. - - Here is an example of a simple call to pcre_dfa_exec(): - - int rc; - int ovector[10]; - int wspace[20]; - rc = pcre_dfa_exec( - re, /* result of pcre_compile() */ - NULL, /* we didn't study the pattern */ - "some string", /* the subject string */ - 11, /* the length of the subject string */ - 0, /* start at offset 0 in the subject */ - 0, /* default options */ - ovector, /* vector of integers for substring information */ - 10, /* number of elements (NOT size in bytes) */ - wspace, /* working space vector */ - 20); /* number of elements (NOT size in bytes) */ - - Option bits for pcre_dfa_exec() - - The unused bits of the options argument for pcre_dfa_exec() must be - zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEW- - LINE_xxx, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, - PCRE_NOTEMPTY_ATSTART, PCRE_NO_UTF8_CHECK, PCRE_BSR_ANYCRLF, - PCRE_BSR_UNICODE, PCRE_NO_START_OPTIMIZE, PCRE_PARTIAL_HARD, PCRE_PAR- - TIAL_SOFT, PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. All but the last - four of these are exactly the same as for pcre_exec(), so their - description is not repeated here. - - PCRE_PARTIAL_HARD - PCRE_PARTIAL_SOFT - - These have the same general effect as they do for pcre_exec(), but the - details are slightly different. When PCRE_PARTIAL_HARD is set for - pcre_dfa_exec(), it returns PCRE_ERROR_PARTIAL if the end of the sub- - ject is reached and there is still at least one matching possibility - that requires additional characters. This happens even if some complete - matches have also been found. When PCRE_PARTIAL_SOFT is set, the return - code PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if the end - of the subject is reached, there have been no complete matches, but - there is still at least one matching possibility. The portion of the - string that was inspected when the longest partial match was found is - set as the first matching string in both cases. There is a more - detailed discussion of partial and multi-segment matching, with exam- - ples, in the pcrepartial documentation. - - PCRE_DFA_SHORTEST - - Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to - stop as soon as it has found one match. Because of the way the alterna- - tive algorithm works, this is necessarily the shortest possible match - at the first possible matching point in the subject string. - - PCRE_DFA_RESTART - - When pcre_dfa_exec() returns a partial match, it is possible to call it - again, with additional subject characters, and have it continue with - the same match. The PCRE_DFA_RESTART option requests this action; when - it is set, the workspace and wscount options must reference the same - vector as before because data about the match so far is left in them - after a partial match. There is more discussion of this facility in the - pcrepartial documentation. - - Successful returns from pcre_dfa_exec() - - When pcre_dfa_exec() succeeds, it may have matched more than one sub- - string in the subject. Note, however, that all the matches from one run - of the function start at the same point in the subject. The shorter - matches are all initial substrings of the longer matches. For example, - if the pattern - - <.*> - - is matched against the string - - This is <something> <something else> <something further> no more - - the three matched strings are - - <something> - <something> <something else> - <something> <something else> <something further> - - On success, the yield of the function is a number greater than zero, - which is the number of matched substrings. The substrings themselves - are returned in ovector. Each string uses two elements; the first is - the offset to the start, and the second is the offset to the end. In - fact, all the strings have the same start offset. (Space could have - been saved by giving this only once, but it was decided to retain some - compatibility with the way pcre_exec() returns data, even though the - meaning of the strings is different.) - - The strings are returned in reverse order of length; that is, the long- - est matching string is given first. If there were too many matches to - fit into ovector, the yield of the function is zero, and the vector is - filled with the longest matches. Unlike pcre_exec(), pcre_dfa_exec() - can use the entire ovector for returning matched strings. - - NOTE: PCRE's "auto-possessification" optimization usually applies to - character repeats at the end of a pattern (as well as internally). For - example, the pattern "a\d+" is compiled as if it were "a\d++" because - there is no point even considering the possibility of backtracking into - the repeated digits. For DFA matching, this means that only one possi- - ble match is found. If you really do want multiple matches in such - cases, either use an ungreedy repeat ("a\d+?") or set the - PCRE_NO_AUTO_POSSESS option when compiling. - - Error returns from pcre_dfa_exec() - - The pcre_dfa_exec() function returns a negative number when it fails. - Many of the errors are the same as for pcre_exec(), and these are - described above. There are in addition the following errors that are - specific to pcre_dfa_exec(): - - PCRE_ERROR_DFA_UITEM (-16) - - This return is given if pcre_dfa_exec() encounters an item in the pat- - tern that it does not support, for instance, the use of \C or a back - reference. - - PCRE_ERROR_DFA_UCOND (-17) - - This return is given if pcre_dfa_exec() encounters a condition item - that uses a back reference for the condition, or a test for recursion - in a specific group. These are not supported. - - PCRE_ERROR_DFA_UMLIMIT (-18) - - This return is given if pcre_dfa_exec() is called with an extra block - that contains a setting of the match_limit or match_limit_recursion - fields. This is not supported (these fields are meaningless for DFA - matching). - - PCRE_ERROR_DFA_WSSIZE (-19) - - This return is given if pcre_dfa_exec() runs out of space in the - workspace vector. - - PCRE_ERROR_DFA_RECURSE (-20) - - When a recursive subpattern is processed, the matching function calls - itself recursively, using private vectors for ovector and workspace. - This error is given if the output vector is not large enough. This - should be extremely rare, as a vector of size 1000 is used. - - PCRE_ERROR_DFA_BADRESTART (-30) - - When pcre_dfa_exec() is called with the PCRE_DFA_RESTART option, some - plausibility checks are made on the contents of the workspace, which - should contain data about the previous partial match. If any of these - checks fail, this error is given. - - -SEE ALSO - - pcre16(3), pcre32(3), pcrebuild(3), pcrecallout(3), pcrecpp(3)(3), - pcrematching(3), pcrepartial(3), pcreposix(3), pcreprecompile(3), pcre- - sample(3), pcrestack(3). - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 09 February 2014 - Copyright (c) 1997-2014 University of Cambridge. ------------------------------------------------------------------------------- - - -PCRECALLOUT(3) Library Functions Manual PCRECALLOUT(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -SYNOPSIS - - #include <pcre.h> - - int (*pcre_callout)(pcre_callout_block *); - - int (*pcre16_callout)(pcre16_callout_block *); - - int (*pcre32_callout)(pcre32_callout_block *); - - -DESCRIPTION - - PCRE provides a feature called "callout", which is a means of temporar- - ily passing control to the caller of PCRE in the middle of pattern - matching. The caller of PCRE provides an external function by putting - its entry point in the global variable pcre_callout (pcre16_callout for - the 16-bit library, pcre32_callout for the 32-bit library). By default, - this variable contains NULL, which disables all calling out. - - Within a regular expression, (?C) indicates the points at which the - external function is to be called. Different callout points can be - identified by putting a number less than 256 after the letter C. The - default value is zero. For example, this pattern has two callout - points: - - (?C1)abc(?C2)def - - If the PCRE_AUTO_CALLOUT option bit is set when a pattern is compiled, - PCRE automatically inserts callouts, all with number 255, before each - item in the pattern. For example, if PCRE_AUTO_CALLOUT is used with the - pattern - - A(\d{2}|--) - - it is processed as if it were - - (?C255)A(?C255)((?C255)\d{2}(?C255)|(?C255)-(?C255)-(?C255))(?C255) - - Notice that there is a callout before and after each parenthesis and - alternation bar. If the pattern contains a conditional group whose con- - dition is an assertion, an automatic callout is inserted immediately - before the condition. Such a callout may also be inserted explicitly, - for example: - - (?(?C9)(?=a)ab|de) - - This applies only to assertion conditions (because they are themselves - independent groups). - - Automatic callouts can be used for tracking the progress of pattern - matching. The pcretest program has a pattern qualifier (/C) that sets - automatic callouts; when it is used, the output indicates how the pat- - tern is being matched. This is useful information when you are trying - to optimize the performance of a particular pattern. - - -MISSING CALLOUTS - - You should be aware that, because of optimizations in the way PCRE com- - piles and matches patterns, callouts sometimes do not happen exactly as - you might expect. - - At compile time, PCRE "auto-possessifies" repeated items when it knows - that what follows cannot be part of the repeat. For example, a+[bc] is - compiled as if it were a++[bc]. The pcretest output when this pattern - is anchored and then applied with automatic callouts to the string - "aaaa" is: - - --->aaaa - +0 ^ ^ - +1 ^ a+ - +3 ^ ^ [bc] - No match - - This indicates that when matching [bc] fails, there is no backtracking - into a+ and therefore the callouts that would be taken for the back- - tracks do not occur. You can disable the auto-possessify feature by - passing PCRE_NO_AUTO_POSSESS to pcre_compile(), or starting the pattern - with (*NO_AUTO_POSSESS). If this is done in pcretest (using the /O - qualifier), the output changes to this: - - --->aaaa - +0 ^ ^ - +1 ^ a+ - +3 ^ ^ [bc] - +3 ^ ^ [bc] - +3 ^ ^ [bc] - +3 ^^ [bc] - No match - - This time, when matching [bc] fails, the matcher backtracks into a+ and - tries again, repeatedly, until a+ itself fails. - - Other optimizations that provide fast "no match" results also affect - callouts. For example, if the pattern is - - ab(?C4)cd - - PCRE knows that any matching string must contain the letter "d". If the - subject string is "abyz", the lack of "d" means that matching doesn't - ever start, and the callout is never reached. However, with "abyd", - though the result is still no match, the callout is obeyed. - - If the pattern is studied, PCRE knows the minimum length of a matching - string, and will immediately give a "no match" return without actually - running a match if the subject is not long enough, or, for unanchored - patterns, if it has been scanned far enough. - - You can disable these optimizations by passing the PCRE_NO_START_OPTI- - MIZE option to the matching function, or by starting the pattern with - (*NO_START_OPT). This slows down the matching process, but does ensure - that callouts such as the example above are obeyed. - - -THE CALLOUT INTERFACE - - During matching, when PCRE reaches a callout point, the external func- - tion defined by pcre_callout or pcre[16|32]_callout is called (if it is - set). This applies to both normal and DFA matching. The only argument - to the callout function is a pointer to a pcre_callout or - pcre[16|32]_callout block. These structures contains the following - fields: - - int version; - int callout_number; - int *offset_vector; - const char *subject; (8-bit version) - PCRE_SPTR16 subject; (16-bit version) - PCRE_SPTR32 subject; (32-bit version) - int subject_length; - int start_match; - int current_position; - int capture_top; - int capture_last; - void *callout_data; - int pattern_position; - int next_item_length; - const unsigned char *mark; (8-bit version) - const PCRE_UCHAR16 *mark; (16-bit version) - const PCRE_UCHAR32 *mark; (32-bit version) - - The version field is an integer containing the version number of the - block format. The initial version was 0; the current version is 2. The - version number will change again in future if additional fields are - added, but the intention is never to remove any of the existing fields. - - The callout_number field contains the number of the callout, as com- - piled into the pattern (that is, the number after ?C for manual call- - outs, and 255 for automatically generated callouts). - - The offset_vector field is a pointer to the vector of offsets that was - passed by the caller to the matching function. When pcre_exec() or - pcre[16|32]_exec() is used, the contents can be inspected, in order to - extract substrings that have been matched so far, in the same way as - for extracting substrings after a match has completed. For the DFA - matching functions, this field is not useful. - - The subject and subject_length fields contain copies of the values that - were passed to the matching function. - - The start_match field normally contains the offset within the subject - at which the current match attempt started. However, if the escape - sequence \K has been encountered, this value is changed to reflect the - modified starting point. If the pattern is not anchored, the callout - function may be called several times from the same point in the pattern - for different starting points in the subject. - - The current_position field contains the offset within the subject of - the current match pointer. - - When the pcre_exec() or pcre[16|32]_exec() is used, the capture_top - field contains one more than the number of the highest numbered cap- - tured substring so far. If no substrings have been captured, the value - of capture_top is one. This is always the case when the DFA functions - are used, because they do not support captured substrings. - - The capture_last field contains the number of the most recently cap- - tured substring. However, when a recursion exits, the value reverts to - what it was outside the recursion, as do the values of all captured - substrings. If no substrings have been captured, the value of cap- - ture_last is -1. This is always the case for the DFA matching func- - tions. - - The callout_data field contains a value that is passed to a matching - function specifically so that it can be passed back in callouts. It is - passed in the callout_data field of a pcre_extra or pcre[16|32]_extra - data structure. If no such data was passed, the value of callout_data - in a callout block is NULL. There is a description of the pcre_extra - structure in the pcreapi documentation. - - The pattern_position field is present from version 1 of the callout - structure. It contains the offset to the next item to be matched in the - pattern string. - - The next_item_length field is present from version 1 of the callout - structure. It contains the length of the next item to be matched in the - pattern string. When the callout immediately precedes an alternation - bar, a closing parenthesis, or the end of the pattern, the length is - zero. When the callout precedes an opening parenthesis, the length is - that of the entire subpattern. - - The pattern_position and next_item_length fields are intended to help - in distinguishing between different automatic callouts, which all have - the same callout number. However, they are set for all callouts. - - The mark field is present from version 2 of the callout structure. In - callouts from pcre_exec() or pcre[16|32]_exec() it contains a pointer - to the zero-terminated name of the most recently passed (*MARK), - (*PRUNE), or (*THEN) item in the match, or NULL if no such items have - been passed. Instances of (*PRUNE) or (*THEN) without a name do not - obliterate a previous (*MARK). In callouts from the DFA matching func- - tions this field always contains NULL. - - -RETURN VALUES - - The external callout function returns an integer to PCRE. If the value - is zero, matching proceeds as normal. If the value is greater than - zero, matching fails at the current point, but the testing of other - matching possibilities goes ahead, just as if a lookahead assertion had - failed. If the value is less than zero, the match is abandoned, the - matching function returns the negative value. - - Negative values should normally be chosen from the set of - PCRE_ERROR_xxx values. In particular, PCRE_ERROR_NOMATCH forces a stan- - dard "no match" failure. The error number PCRE_ERROR_CALLOUT is - reserved for use by callout functions; it will never be used by PCRE - itself. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 12 November 2013 - Copyright (c) 1997-2013 University of Cambridge. ------------------------------------------------------------------------------- - - -PCRECOMPAT(3) Library Functions Manual PCRECOMPAT(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -DIFFERENCES BETWEEN PCRE AND PERL - - This document describes the differences in the ways that PCRE and Perl - handle regular expressions. The differences described here are with - respect to Perl versions 5.10 and above. - - 1. PCRE has only a subset of Perl's Unicode support. Details of what it - does have are given in the pcreunicode page. - - 2. PCRE allows repeat quantifiers only on parenthesized assertions, but - they do not mean what you might think. For example, (?!a){3} does not - assert that the next three characters are not "a". It just asserts that - the next character is not "a" three times (in principle: PCRE optimizes - this to run the assertion just once). Perl allows repeat quantifiers on - other assertions such as \b, but these do not seem to have any use. - - 3. Capturing subpatterns that occur inside negative lookahead asser- - tions are counted, but their entries in the offsets vector are never - set. Perl sometimes (but not always) sets its numerical variables from - inside negative assertions. - - 4. Though binary zero characters are supported in the subject string, - they are not allowed in a pattern string because it is passed as a nor- - mal C string, terminated by zero. The escape sequence \0 can be used in - the pattern to represent a binary zero. - - 5. The following Perl escape sequences are not supported: \l, \u, \L, - \U, and \N when followed by a character name or Unicode value. (\N on - its own, matching a non-newline character, is supported.) In fact these - are implemented by Perl's general string-handling and are not part of - its pattern matching engine. If any of these are encountered by PCRE, - an error is generated by default. However, if the PCRE_JAVASCRIPT_COM- - PAT option is set, \U and \u are interpreted as JavaScript interprets - them. - - 6. The Perl escape sequences \p, \P, and \X are supported only if PCRE - is built with Unicode character property support. The properties that - can be tested with \p and \P are limited to the general category prop- - erties such as Lu and Nd, script names such as Greek or Han, and the - derived properties Any and L&. PCRE does support the Cs (surrogate) - property, which Perl does not; the Perl documentation says "Because - Perl hides the need for the user to understand the internal representa- - tion of Unicode characters, there is no need to implement the somewhat - messy concept of surrogates." - - 7. PCRE does support the \Q...\E escape for quoting substrings. Charac- - ters in between are treated as literals. This is slightly different - from Perl in that $ and @ are also handled as literals inside the - quotes. In Perl, they cause variable interpolation (but of course PCRE - does not have variables). Note the following examples: - - Pattern PCRE matches Perl matches - - \Qabc$xyz\E abc$xyz abc followed by the - contents of $xyz - \Qabc\$xyz\E abc\$xyz abc\$xyz - \Qabc\E\$\Qxyz\E abc$xyz abc$xyz - - The \Q...\E sequence is recognized both inside and outside character - classes. - - 8. Fairly obviously, PCRE does not support the (?{code}) and (??{code}) - constructions. However, there is support for recursive patterns. This - is not available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE - "callout" feature allows an external function to be called during pat- - tern matching. See the pcrecallout documentation for details. - - 9. Subpatterns that are called as subroutines (whether or not recur- - sively) are always treated as atomic groups in PCRE. This is like - Python, but unlike Perl. Captured values that are set outside a sub- - routine call can be reference from inside in PCRE, but not in Perl. - There is a discussion that explains these differences in more detail in - the section on recursion differences from Perl in the pcrepattern page. - - 10. If any of the backtracking control verbs are used in a subpattern - that is called as a subroutine (whether or not recursively), their - effect is confined to that subpattern; it does not extend to the sur- - rounding pattern. This is not always the case in Perl. In particular, - if (*THEN) is present in a group that is called as a subroutine, its - action is limited to that group, even if the group does not contain any - | characters. Note that such subpatterns are processed as anchored at - the point where they are tested. - - 11. If a pattern contains more than one backtracking control verb, the - first one that is backtracked onto acts. For example, in the pattern - A(*COMMIT)B(*PRUNE)C a failure in B triggers (*COMMIT), but a failure - in C triggers (*PRUNE). Perl's behaviour is more complex; in many cases - it is the same as PCRE, but there are examples where it differs. - - 12. Most backtracking verbs in assertions have their normal actions. - They are not confined to the assertion. - - 13. There are some differences that are concerned with the settings of - captured strings when part of a pattern is repeated. For example, - matching "aba" against the pattern /^(a(b)?)+$/ in Perl leaves $2 - unset, but in PCRE it is set to "b". - - 14. PCRE's handling of duplicate subpattern numbers and duplicate sub- - pattern names is not as general as Perl's. This is a consequence of the - fact the PCRE works internally just with numbers, using an external ta- - ble to translate between numbers and names. In particular, a pattern - such as (?|(?<a>A)|(?<b)B), where the two capturing parentheses have - the same number but different names, is not supported, and causes an - error at compile time. If it were allowed, it would not be possible to - distinguish which parentheses matched, because both names map to cap- - turing subpattern number 1. To avoid this confusing situation, an error - is given at compile time. - - 15. Perl recognizes comments in some places that PCRE does not, for - example, between the ( and ? at the start of a subpattern. If the /x - modifier is set, Perl allows white space between ( and ? (though cur- - rent Perls warn that this is deprecated) but PCRE never does, even if - the PCRE_EXTENDED option is set. - - 16. Perl, when in warning mode, gives warnings for character classes - such as [A-\d] or [a-[:digit:]]. It then treats the hyphens as liter- - als. PCRE has no warning features, so it gives an error in these cases - because they are almost certainly user mistakes. - - 17. In PCRE, the upper/lower case character properties Lu and Ll are - not affected when case-independent matching is specified. For example, - \p{Lu} always matches an upper case letter. I think Perl has changed in - this respect; in the release at the time of writing (5.16), \p{Lu} and - \p{Ll} match all letters, regardless of case, when case independence is - specified. - - 18. PCRE provides some extensions to the Perl regular expression facil- - ities. Perl 5.10 includes new features that are not in earlier ver- - sions of Perl, some of which (such as named parentheses) have been in - PCRE for some time. This list is with respect to Perl 5.10: - - (a) Although lookbehind assertions in PCRE must match fixed length - strings, each alternative branch of a lookbehind assertion can match a - different length of string. Perl requires them all to have the same - length. - - (b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $ - meta-character matches only at the very end of the string. - - (c) If PCRE_EXTRA is set, a backslash followed by a letter with no spe- - cial meaning is faulted. Otherwise, like Perl, the backslash is quietly - ignored. (Perl can be made to issue a warning.) - - (d) If PCRE_UNGREEDY is set, the greediness of the repetition quanti- - fiers is inverted, that is, by default they are not greedy, but if fol- - lowed by a question mark they are. - - (e) PCRE_ANCHORED can be used at matching time to force a pattern to be - tried only at the first matching position in the subject string. - - (f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, - and PCRE_NO_AUTO_CAPTURE options for pcre_exec() have no Perl equiva- - lents. - - (g) The \R escape sequence can be restricted to match only CR, LF, or - CRLF by the PCRE_BSR_ANYCRLF option. - - (h) The callout facility is PCRE-specific. - - (i) The partial matching facility is PCRE-specific. - - (j) Patterns compiled by PCRE can be saved and re-used at a later time, - even on different hosts that have the other endianness. However, this - does not apply to optimized data created by the just-in-time compiler. - - (k) The alternative matching functions (pcre_dfa_exec(), - pcre16_dfa_exec() and pcre32_dfa_exec(),) match in a different way and - are not Perl-compatible. - - (l) PCRE recognizes some special sequences such as (*CR) at the start - of a pattern that set overall options that cannot be changed within the - pattern. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 10 November 2013 - Copyright (c) 1997-2013 University of Cambridge. ------------------------------------------------------------------------------- - - -PCREPATTERN(3) Library Functions Manual PCREPATTERN(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -PCRE REGULAR EXPRESSION DETAILS - - The syntax and semantics of the regular expressions that are supported - by PCRE are described in detail below. There is a quick-reference syn- - tax summary in the pcresyntax page. PCRE tries to match Perl syntax and - semantics as closely as it can. PCRE also supports some alternative - regular expression syntax (which does not conflict with the Perl syn- - tax) in order to provide some compatibility with regular expressions in - Python, .NET, and Oniguruma. - - Perl's regular expressions are described in its own documentation, and - regular expressions in general are covered in a number of books, some - of which have copious examples. Jeffrey Friedl's "Mastering Regular - Expressions", published by O'Reilly, covers regular expressions in - great detail. This description of PCRE's regular expressions is - intended as reference material. - - This document discusses the patterns that are supported by PCRE when - one its main matching functions, pcre_exec() (8-bit) or - pcre[16|32]_exec() (16- or 32-bit), is used. PCRE also has alternative - matching functions, pcre_dfa_exec() and pcre[16|32_dfa_exec(), which - match using a different algorithm that is not Perl-compatible. Some of - the features discussed below are not available when DFA matching is - used. The advantages and disadvantages of the alternative functions, - and how they differ from the normal functions, are discussed in the - pcrematching page. - - -SPECIAL START-OF-PATTERN ITEMS - - A number of options that can be passed to pcre_compile() can also be - set by special items at the start of a pattern. These are not Perl-com- - patible, but are provided to make these options accessible to pattern - writers who are not able to change the program that processes the pat- - tern. Any number of these items may appear, but they must all be - together right at the start of the pattern string, and the letters must - be in upper case. - - UTF support - - The original operation of PCRE was on strings of one-byte characters. - However, there is now also support for UTF-8 strings in the original - library, an extra library that supports 16-bit and UTF-16 character - strings, and a third library that supports 32-bit and UTF-32 character - strings. To use these features, PCRE must be built to include appropri- - ate support. When using UTF strings you must either call the compiling - function with the PCRE_UTF8, PCRE_UTF16, or PCRE_UTF32 option, or the - pattern must start with one of these special sequences: - - (*UTF8) - (*UTF16) - (*UTF32) - (*UTF) - - (*UTF) is a generic sequence that can be used with any of the - libraries. Starting a pattern with such a sequence is equivalent to - setting the relevant option. How setting a UTF mode affects pattern - matching is mentioned in several places below. There is also a summary - of features in the pcreunicode page. - - Some applications that allow their users to supply patterns may wish to - restrict them to non-UTF data for security reasons. If the - PCRE_NEVER_UTF option is set at compile time, (*UTF) etc. are not - allowed, and their appearance causes an error. - - Unicode property support - - Another special sequence that may appear at the start of a pattern is - (*UCP). This has the same effect as setting the PCRE_UCP option: it - causes sequences such as \d and \w to use Unicode properties to deter- - mine character types, instead of recognizing only characters with codes - less than 128 via a lookup table. - - Disabling auto-possessification - - If a pattern starts with (*NO_AUTO_POSSESS), it has the same effect as - setting the PCRE_NO_AUTO_POSSESS option at compile time. This stops - PCRE from making quantifiers possessive when what follows cannot match - the repeated item. For example, by default a+b is treated as a++b. For - more details, see the pcreapi documentation. - - Disabling start-up optimizations - - If a pattern starts with (*NO_START_OPT), it has the same effect as - setting the PCRE_NO_START_OPTIMIZE option either at compile or matching - time. This disables several optimizations for quickly reaching "no - match" results. For more details, see the pcreapi documentation. - - Newline conventions - - PCRE supports five different conventions for indicating line breaks in - strings: a single CR (carriage return) character, a single LF (line- - feed) character, the two-character sequence CRLF, any of the three pre- - ceding, or any Unicode newline sequence. The pcreapi page has further - discussion about newlines, and shows how to set the newline convention - in the options arguments for the compiling and matching functions. - - It is also possible to specify a newline convention by starting a pat- - tern string with one of the following five sequences: - - (*CR) carriage return - (*LF) linefeed - (*CRLF) carriage return, followed by linefeed - (*ANYCRLF) any of the three above - (*ANY) all Unicode newline sequences - - These override the default and the options given to the compiling func- - tion. For example, on a Unix system where LF is the default newline - sequence, the pattern - - (*CR)a.b - - changes the convention to CR. That pattern matches "a\nb" because LF is - no longer a newline. If more than one of these settings is present, the - last one is used. - - The newline convention affects where the circumflex and dollar asser- - tions are true. It also affects the interpretation of the dot metachar- - acter when PCRE_DOTALL is not set, and the behaviour of \N. However, it - does not affect what the \R escape sequence matches. By default, this - is any Unicode newline sequence, for Perl compatibility. However, this - can be changed; see the description of \R in the section entitled "New- - line sequences" below. A change of \R setting can be combined with a - change of newline convention. - - Setting match and recursion limits - - The caller of pcre_exec() can set a limit on the number of times the - internal match() function is called and on the maximum depth of recur- - sive calls. These facilities are provided to catch runaway matches that - are provoked by patterns with huge matching trees (a typical example is - a pattern with nested unlimited repeats) and to avoid running out of - system stack by too much recursion. When one of these limits is - reached, pcre_exec() gives an error return. The limits can also be set - by items at the start of the pattern of the form - - (*LIMIT_MATCH=d) - (*LIMIT_RECURSION=d) - - where d is any number of decimal digits. However, the value of the set- - ting must be less than the value set (or defaulted) by the caller of - pcre_exec() for it to have any effect. In other words, the pattern - writer can lower the limits set by the programmer, but not raise them. - If there is more than one setting of one of these limits, the lower - value is used. - - -EBCDIC CHARACTER CODES - - PCRE can be compiled to run in an environment that uses EBCDIC as its - character code rather than ASCII or Unicode (typically a mainframe sys- - tem). In the sections below, character code values are ASCII or Uni- - code; in an EBCDIC environment these characters may have different code - values, and there are no code points greater than 255. - - -CHARACTERS AND METACHARACTERS - - A regular expression is a pattern that is matched against a subject - string from left to right. Most characters stand for themselves in a - pattern, and match the corresponding characters in the subject. As a - trivial example, the pattern - - The quick brown fox - - matches a portion of a subject string that is identical to itself. When - caseless matching is specified (the PCRE_CASELESS option), letters are - matched independently of case. In a UTF mode, PCRE always understands - the concept of case for characters whose values are less than 128, so - caseless matching is always possible. For characters with higher val- - ues, the concept of case is supported if PCRE is compiled with Unicode - property support, but not otherwise. If you want to use caseless - matching for characters 128 and above, you must ensure that PCRE is - compiled with Unicode property support as well as with UTF support. - - The power of regular expressions comes from the ability to include - alternatives and repetitions in the pattern. These are encoded in the - pattern by the use of metacharacters, which do not stand for themselves - but instead are interpreted in some special way. - - There are two different sets of metacharacters: those that are recog- - nized anywhere in the pattern except within square brackets, and those - that are recognized within square brackets. Outside square brackets, - the metacharacters are as follows: - - \ general escape character with several uses - ^ assert start of string (or line, in multiline mode) - $ assert end of string (or line, in multiline mode) - . match any character except newline (by default) - [ start character class definition - | start of alternative branch - ( start subpattern - ) end subpattern - ? extends the meaning of ( - also 0 or 1 quantifier - also quantifier minimizer - * 0 or more quantifier - + 1 or more quantifier - also "possessive quantifier" - { start min/max quantifier - - Part of a pattern that is in square brackets is called a "character - class". In a character class the only metacharacters are: - - \ general escape character - ^ negate the class, but only if the first character - - indicates character range - [ POSIX character class (only if followed by POSIX - syntax) - ] terminates the character class - - The following sections describe the use of each of the metacharacters. - - -BACKSLASH - - The backslash character has several uses. Firstly, if it is followed by - a character that is not a number or a letter, it takes away any special - meaning that character may have. This use of backslash as an escape - character applies both inside and outside character classes. - - For example, if you want to match a * character, you write \* in the - pattern. This escaping action applies whether or not the following - character would otherwise be interpreted as a metacharacter, so it is - always safe to precede a non-alphanumeric with backslash to specify - that it stands for itself. In particular, if you want to match a back- - slash, you write \\. - - In a UTF mode, only ASCII numbers and letters have any special meaning - after a backslash. All other characters (in particular, those whose - codepoints are greater than 127) are treated as literals. - - If a pattern is compiled with the PCRE_EXTENDED option, most white - space in the pattern (other than in a character class), and characters - between a # outside a character class and the next newline, inclusive, - are ignored. An escaping backslash can be used to include a white space - or # character as part of the pattern. - - If you want to remove the special meaning from a sequence of charac- - ters, you can do so by putting them between \Q and \E. This is differ- - ent from Perl in that $ and @ are handled as literals in \Q...\E - sequences in PCRE, whereas in Perl, $ and @ cause variable interpola- - tion. Note the following examples: - - Pattern PCRE matches Perl matches - - \Qabc$xyz\E abc$xyz abc followed by the - contents of $xyz - \Qabc\$xyz\E abc\$xyz abc\$xyz - \Qabc\E\$\Qxyz\E abc$xyz abc$xyz - - The \Q...\E sequence is recognized both inside and outside character - classes. An isolated \E that is not preceded by \Q is ignored. If \Q - is not followed by \E later in the pattern, the literal interpretation - continues to the end of the pattern (that is, \E is assumed at the - end). If the isolated \Q is inside a character class, this causes an - error, because the character class is not terminated. - - Non-printing characters - - A second use of backslash provides a way of encoding non-printing char- - acters in patterns in a visible manner. There is no restriction on the - appearance of non-printing characters, apart from the binary zero that - terminates a pattern, but when a pattern is being prepared by text - editing, it is often easier to use one of the following escape - sequences than the binary character it represents: - - \a alarm, that is, the BEL character (hex 07) - \cx "control-x", where x is any ASCII character - \e escape (hex 1B) - \f form feed (hex 0C) - \n linefeed (hex 0A) - \r carriage return (hex 0D) - \t tab (hex 09) - \0dd character with octal code 0dd - \ddd character with octal code ddd, or back reference - \o{ddd..} character with octal code ddd.. - \xhh character with hex code hh - \x{hhh..} character with hex code hhh.. (non-JavaScript mode) - \uhhhh character with hex code hhhh (JavaScript mode only) - - The precise effect of \cx on ASCII characters is as follows: if x is a - lower case letter, it is converted to upper case. Then bit 6 of the - character (hex 40) is inverted. Thus \cA to \cZ become hex 01 to hex 1A - (A is 41, Z is 5A), but \c{ becomes hex 3B ({ is 7B), and \c; becomes - hex 7B (; is 3B). If the data item (byte or 16-bit value) following \c - has a value greater than 127, a compile-time error occurs. This locks - out non-ASCII characters in all modes. - - The \c facility was designed for use with ASCII characters, but with - the extension to Unicode it is even less useful than it once was. It - is, however, recognized when PCRE is compiled in EBCDIC mode, where - data items are always bytes. In this mode, all values are valid after - \c. If the next character is a lower case letter, it is converted to - upper case. Then the 0xc0 bits of the byte are inverted. Thus \cA - becomes hex 01, as in ASCII (A is C1), but because the EBCDIC letters - are disjoint, \cZ becomes hex 29 (Z is E9), and other characters also - generate different values. - - After \0 up to two further octal digits are read. If there are fewer - than two digits, just those that are present are used. Thus the - sequence \0\x\07 specifies two binary zeros followed by a BEL character - (code value 7). Make sure you supply two digits after the initial zero - if the pattern character that follows is itself an octal digit. - - The escape \o must be followed by a sequence of octal digits, enclosed - in braces. An error occurs if this is not the case. This escape is a - recent addition to Perl; it provides way of specifying character code - points as octal numbers greater than 0777, and it also allows octal - numbers and back references to be unambiguously specified. - - For greater clarity and unambiguity, it is best to avoid following \ by - a digit greater than zero. Instead, use \o{} or \x{} to specify charac- - ter numbers, and \g{} to specify back references. The following para- - graphs describe the old, ambiguous syntax. - - The handling of a backslash followed by a digit other than 0 is compli- - cated, and Perl has changed in recent releases, causing PCRE also to - change. Outside a character class, PCRE reads the digit and any follow- - ing digits as a decimal number. If the number is less than 8, or if - there have been at least that many previous capturing left parentheses - in the expression, the entire sequence is taken as a back reference. A - description of how this works is given later, following the discussion - of parenthesized subpatterns. - - Inside a character class, or if the decimal number following \ is - greater than 7 and there have not been that many capturing subpatterns, - PCRE handles \8 and \9 as the literal characters "8" and "9", and oth- - erwise re-reads up to three octal digits following the backslash, using - them to generate a data character. Any subsequent digits stand for - themselves. For example: - - \040 is another way of writing an ASCII space - \40 is the same, provided there are fewer than 40 - previous capturing subpatterns - \7 is always a back reference - \11 might be a back reference, or another way of - writing a tab - \011 is always a tab - \0113 is a tab followed by the character "3" - \113 might be a back reference, otherwise the - character with octal code 113 - \377 might be a back reference, otherwise - the value 255 (decimal) - \81 is either a back reference, or the two - characters "8" and "1" - - Note that octal values of 100 or greater that are specified using this - syntax must not be introduced by a leading zero, because no more than - three octal digits are ever read. - - By default, after \x that is not followed by {, from zero to two hexa- - decimal digits are read (letters can be in upper or lower case). Any - number of hexadecimal digits may appear between \x{ and }. If a charac- - ter other than a hexadecimal digit appears between \x{ and }, or if - there is no terminating }, an error occurs. - - If the PCRE_JAVASCRIPT_COMPAT option is set, the interpretation of \x - is as just described only when it is followed by two hexadecimal dig- - its. Otherwise, it matches a literal "x" character. In JavaScript - mode, support for code points greater than 256 is provided by \u, which - must be followed by four hexadecimal digits; otherwise it matches a - literal "u" character. - - Characters whose value is less than 256 can be defined by either of the - two syntaxes for \x (or by \u in JavaScript mode). There is no differ- - ence in the way they are handled. For example, \xdc is exactly the same - as \x{dc} (or \u00dc in JavaScript mode). - - Constraints on character values - - Characters that are specified using octal or hexadecimal numbers are - limited to certain values, as follows: - - 8-bit non-UTF mode less than 0x100 - 8-bit UTF-8 mode less than 0x10ffff and a valid codepoint - 16-bit non-UTF mode less than 0x10000 - 16-bit UTF-16 mode less than 0x10ffff and a valid codepoint - 32-bit non-UTF mode less than 0x100000000 - 32-bit UTF-32 mode less than 0x10ffff and a valid codepoint - - Invalid Unicode codepoints are the range 0xd800 to 0xdfff (the so- - called "surrogate" codepoints), and 0xffef. - - Escape sequences in character classes - - All the sequences that define a single character value can be used both - inside and outside character classes. In addition, inside a character - class, \b is interpreted as the backspace character (hex 08). - - \N is not allowed in a character class. \B, \R, and \X are not special - inside a character class. Like other unrecognized escape sequences, - they are treated as the literal characters "B", "R", and "X" by - default, but cause an error if the PCRE_EXTRA option is set. Outside a - character class, these sequences have different meanings. - - Unsupported escape sequences - - In Perl, the sequences \l, \L, \u, and \U are recognized by its string - handler and used to modify the case of following characters. By - default, PCRE does not support these escape sequences. However, if the - PCRE_JAVASCRIPT_COMPAT option is set, \U matches a "U" character, and - \u can be used to define a character by code point, as described in the - previous section. - - Absolute and relative back references - - The sequence \g followed by an unsigned or a negative number, option- - ally enclosed in braces, is an absolute or relative back reference. A - named back reference can be coded as \g{name}. Back references are dis- - cussed later, following the discussion of parenthesized subpatterns. - - Absolute and relative subroutine calls - - For compatibility with Oniguruma, the non-Perl syntax \g followed by a - name or a number enclosed either in angle brackets or single quotes, is - an alternative syntax for referencing a subpattern as a "subroutine". - Details are discussed later. Note that \g{...} (Perl syntax) and - \g<...> (Oniguruma syntax) are not synonymous. The former is a back - reference; the latter is a subroutine call. - - Generic character types - - Another use of backslash is for specifying generic character types: - - \d any decimal digit - \D any character that is not a decimal digit - \h any horizontal white space character - \H any character that is not a horizontal white space character - \s any white space character - \S any character that is not a white space character - \v any vertical white space character - \V any character that is not a vertical white space character - \w any "word" character - \W any "non-word" character - - There is also the single sequence \N, which matches a non-newline char- - acter. This is the same as the "." metacharacter when PCRE_DOTALL is - not set. Perl also uses \N to match characters by name; PCRE does not - support this. - - Each pair of lower and upper case escape sequences partitions the com- - plete set of characters into two disjoint sets. Any given character - matches one, and only one, of each pair. The sequences can appear both - inside and outside character classes. They each match one character of - the appropriate type. If the current matching point is at the end of - the subject string, all of them fail, because there is no character to - match. - - For compatibility with Perl, \s did not used to match the VT character - (code 11), which made it different from the the POSIX "space" class. - However, Perl added VT at release 5.18, and PCRE followed suit at - release 8.34. The default \s characters are now HT (9), LF (10), VT - (11), FF (12), CR (13), and space (32), which are defined as white - space in the "C" locale. This list may vary if locale-specific matching - is taking place. For example, in some locales the "non-breaking space" - character (\xA0) is recognized as white space, and in others the VT - character is not. - - A "word" character is an underscore or any character that is a letter - or digit. By default, the definition of letters and digits is con- - trolled by PCRE's low-valued character tables, and may vary if locale- - specific matching is taking place (see "Locale support" in the pcreapi - page). For example, in a French locale such as "fr_FR" in Unix-like - systems, or "french" in Windows, some character codes greater than 127 - are used for accented letters, and these are then matched by \w. The - use of locales with Unicode is discouraged. - - By default, characters whose code points are greater than 127 never - match \d, \s, or \w, and always match \D, \S, and \W, although this may - vary for characters in the range 128-255 when locale-specific matching - is happening. These escape sequences retain their original meanings - from before Unicode support was available, mainly for efficiency rea- - sons. If PCRE is compiled with Unicode property support, and the - PCRE_UCP option is set, the behaviour is changed so that Unicode prop- - erties are used to determine character types, as follows: - - \d any character that matches \p{Nd} (decimal digit) - \s any character that matches \p{Z} or \h or \v - \w any character that matches \p{L} or \p{N}, plus underscore - - The upper case escapes match the inverse sets of characters. Note that - \d matches only decimal digits, whereas \w matches any Unicode digit, - as well as any Unicode letter, and underscore. Note also that PCRE_UCP - affects \b, and \B because they are defined in terms of \w and \W. - Matching these sequences is noticeably slower when PCRE_UCP is set. - - The sequences \h, \H, \v, and \V are features that were added to Perl - at release 5.10. In contrast to the other sequences, which match only - ASCII characters by default, these always match certain high-valued - code points, whether or not PCRE_UCP is set. The horizontal space char- - acters are: - - U+0009 Horizontal tab (HT) - U+0020 Space - U+00A0 Non-break space - U+1680 Ogham space mark - U+180E Mongolian vowel separator - U+2000 En quad - U+2001 Em quad - U+2002 En space - U+2003 Em space - U+2004 Three-per-em space - U+2005 Four-per-em space - U+2006 Six-per-em space - U+2007 Figure space - U+2008 Punctuation space - U+2009 Thin space - U+200A Hair space - U+202F Narrow no-break space - U+205F Medium mathematical space - U+3000 Ideographic space - - The vertical space characters are: - - U+000A Linefeed (LF) - U+000B Vertical tab (VT) - U+000C Form feed (FF) - U+000D Carriage return (CR) - U+0085 Next line (NEL) - U+2028 Line separator - U+2029 Paragraph separator - - In 8-bit, non-UTF-8 mode, only the characters with codepoints less than - 256 are relevant. - - Newline sequences - - Outside a character class, by default, the escape sequence \R matches - any Unicode newline sequence. In 8-bit non-UTF-8 mode \R is equivalent - to the following: - - (?>\r\n|\n|\x0b|\f|\r|\x85) - - This is an example of an "atomic group", details of which are given - below. This particular group matches either the two-character sequence - CR followed by LF, or one of the single characters LF (linefeed, - U+000A), VT (vertical tab, U+000B), FF (form feed, U+000C), CR (car- - riage return, U+000D), or NEL (next line, U+0085). The two-character - sequence is treated as a single unit that cannot be split. - - In other modes, two additional characters whose codepoints are greater - than 255 are added: LS (line separator, U+2028) and PS (paragraph sepa- - rator, U+2029). Unicode character property support is not needed for - these characters to be recognized. - - It is possible to restrict \R to match only CR, LF, or CRLF (instead of - the complete set of Unicode line endings) by setting the option - PCRE_BSR_ANYCRLF either at compile time or when the pattern is matched. - (BSR is an abbrevation for "backslash R".) This can be made the default - when PCRE is built; if this is the case, the other behaviour can be - requested via the PCRE_BSR_UNICODE option. It is also possible to - specify these settings by starting a pattern string with one of the - following sequences: - - (*BSR_ANYCRLF) CR, LF, or CRLF only - (*BSR_UNICODE) any Unicode newline sequence - - These override the default and the options given to the compiling func- - tion, but they can themselves be overridden by options given to a - matching function. Note that these special settings, which are not - Perl-compatible, are recognized only at the very start of a pattern, - and that they must be in upper case. If more than one of them is - present, the last one is used. They can be combined with a change of - newline convention; for example, a pattern can start with: - - (*ANY)(*BSR_ANYCRLF) - - They can also be combined with the (*UTF8), (*UTF16), (*UTF32), (*UTF) - or (*UCP) special sequences. Inside a character class, \R is treated as - an unrecognized escape sequence, and so matches the letter "R" by - default, but causes an error if PCRE_EXTRA is set. - - Unicode character properties - - When PCRE is built with Unicode character property support, three addi- - tional escape sequences that match characters with specific properties - are available. When in 8-bit non-UTF-8 mode, these sequences are of - course limited to testing characters whose codepoints are less than - 256, but they do work in this mode. The extra escape sequences are: - - \p{xx} a character with the xx property - \P{xx} a character without the xx property - \X a Unicode extended grapheme cluster - - The property names represented by xx above are limited to the Unicode - script names, the general category properties, "Any", which matches any - character (including newline), and some special PCRE properties - (described in the next section). Other Perl properties such as "InMu- - sicalSymbols" are not currently supported by PCRE. Note that \P{Any} - does not match any characters, so always causes a match failure. - - Sets of Unicode characters are defined as belonging to certain scripts. - A character from one of these sets can be matched using a script name. - For example: - - \p{Greek} - \P{Han} - - Those that are not part of an identified script are lumped together as - "Common". The current list of scripts is: - - Arabic, Armenian, Avestan, Balinese, Bamum, Batak, Bengali, Bopomofo, - Brahmi, Braille, Buginese, Buhid, Canadian_Aboriginal, Carian, Chakma, - Cham, Cherokee, Common, Coptic, Cuneiform, Cypriot, Cyrillic, Deseret, - Devanagari, Egyptian_Hieroglyphs, Ethiopic, Georgian, Glagolitic, - Gothic, Greek, Gujarati, Gurmukhi, Han, Hangul, Hanunoo, Hebrew, Hira- - gana, Imperial_Aramaic, Inherited, Inscriptional_Pahlavi, Inscrip- - tional_Parthian, Javanese, Kaithi, Kannada, Katakana, Kayah_Li, - Kharoshthi, Khmer, Lao, Latin, Lepcha, Limbu, Linear_B, Lisu, Lycian, - Lydian, Malayalam, Mandaic, Meetei_Mayek, Meroitic_Cursive, - Meroitic_Hieroglyphs, Miao, Mongolian, Myanmar, New_Tai_Lue, Nko, - Ogham, Old_Italic, Old_Persian, Old_South_Arabian, Old_Turkic, - Ol_Chiki, Oriya, Osmanya, Phags_Pa, Phoenician, Rejang, Runic, Samari- - tan, Saurashtra, Sharada, Shavian, Sinhala, Sora_Sompeng, Sundanese, - Syloti_Nagri, Syriac, Tagalog, Tagbanwa, Tai_Le, Tai_Tham, Tai_Viet, - Takri, Tamil, Telugu, Thaana, Thai, Tibetan, Tifinagh, Ugaritic, Vai, - Yi. - - Each character has exactly one Unicode general category property, spec- - ified by a two-letter abbreviation. For compatibility with Perl, nega- - tion can be specified by including a circumflex between the opening - brace and the property name. For example, \p{^Lu} is the same as - \P{Lu}. - - If only one letter is specified with \p or \P, it includes all the gen- - eral category properties that start with that letter. In this case, in - the absence of negation, the curly brackets in the escape sequence are - optional; these two examples have the same effect: - - \p{L} - \pL - - The following general category property codes are supported: - - C Other - Cc Control - Cf Format - Cn Unassigned - Co Private use - Cs Surrogate - - L Letter - Ll Lower case letter - Lm Modifier letter - Lo Other letter - Lt Title case letter - Lu Upper case letter - - M Mark - Mc Spacing mark - Me Enclosing mark - Mn Non-spacing mark - - N Number - Nd Decimal number - Nl Letter number - No Other number - - P Punctuation - Pc Connector punctuation - Pd Dash punctuation - Pe Close punctuation - Pf Final punctuation - Pi Initial punctuation - Po Other punctuation - Ps Open punctuation - - S Symbol - Sc Currency symbol - Sk Modifier symbol - Sm Mathematical symbol - So Other symbol - - Z Separator - Zl Line separator - Zp Paragraph separator - Zs Space separator - - The special property L& is also supported: it matches a character that - has the Lu, Ll, or Lt property, in other words, a letter that is not - classified as a modifier or "other". - - The Cs (Surrogate) property applies only to characters in the range - U+D800 to U+DFFF. Such characters are not valid in Unicode strings and - so cannot be tested by PCRE, unless UTF validity checking has been - turned off (see the discussion of PCRE_NO_UTF8_CHECK, - PCRE_NO_UTF16_CHECK and PCRE_NO_UTF32_CHECK in the pcreapi page). Perl - does not support the Cs property. - - The long synonyms for property names that Perl supports (such as - \p{Letter}) are not supported by PCRE, nor is it permitted to prefix - any of these properties with "Is". - - No character that is in the Unicode table has the Cn (unassigned) prop- - erty. Instead, this property is assumed for any code point that is not - in the Unicode table. - - Specifying caseless matching does not affect these escape sequences. - For example, \p{Lu} always matches only upper case letters. This is - different from the behaviour of current versions of Perl. - - Matching characters by Unicode property is not fast, because PCRE has - to do a multistage table lookup in order to find a character's prop- - erty. That is why the traditional escape sequences such as \d and \w do - not use Unicode properties in PCRE by default, though you can make them - do so by setting the PCRE_UCP option or by starting the pattern with - (*UCP). - - Extended grapheme clusters - - The \X escape matches any number of Unicode characters that form an - "extended grapheme cluster", and treats the sequence as an atomic group - (see below). Up to and including release 8.31, PCRE matched an ear- - lier, simpler definition that was equivalent to - - (?>\PM\pM*) - - That is, it matched a character without the "mark" property, followed - by zero or more characters with the "mark" property. Characters with - the "mark" property are typically non-spacing accents that affect the - preceding character. - - This simple definition was extended in Unicode to include more compli- - cated kinds of composite character by giving each character a grapheme - breaking property, and creating rules that use these properties to - define the boundaries of extended grapheme clusters. In releases of - PCRE later than 8.31, \X matches one of these clusters. - - \X always matches at least one character. Then it decides whether to - add additional characters according to the following rules for ending a - cluster: - - 1. End at the end of the subject string. - - 2. Do not end between CR and LF; otherwise end after any control char- - acter. - - 3. Do not break Hangul (a Korean script) syllable sequences. Hangul - characters are of five types: L, V, T, LV, and LVT. An L character may - be followed by an L, V, LV, or LVT character; an LV or V character may - be followed by a V or T character; an LVT or T character may be follwed - only by a T character. - - 4. Do not end before extending characters or spacing marks. Characters - with the "mark" property always have the "extend" grapheme breaking - property. - - 5. Do not end after prepend characters. - - 6. Otherwise, end the cluster. - - PCRE's additional properties - - As well as the standard Unicode properties described above, PCRE sup- - ports four more that make it possible to convert traditional escape - sequences such as \w and \s to use Unicode properties. PCRE uses these - non-standard, non-Perl properties internally when PCRE_UCP is set. How- - ever, they may also be used explicitly. These properties are: - - Xan Any alphanumeric character - Xps Any POSIX space character - Xsp Any Perl space character - Xwd Any Perl "word" character - - Xan matches characters that have either the L (letter) or the N (num- - ber) property. Xps matches the characters tab, linefeed, vertical tab, - form feed, or carriage return, and any other character that has the Z - (separator) property. Xsp is the same as Xps; it used to exclude ver- - tical tab, for Perl compatibility, but Perl changed, and so PCRE fol- - lowed at release 8.34. Xwd matches the same characters as Xan, plus - underscore. - - There is another non-standard property, Xuc, which matches any charac- - ter that can be represented by a Universal Character Name in C++ and - other programming languages. These are the characters $, @, ` (grave - accent), and all characters with Unicode code points greater than or - equal to U+00A0, except for the surrogates U+D800 to U+DFFF. Note that - most base (ASCII) characters are excluded. (Universal Character Names - are of the form \uHHHH or \UHHHHHHHH where H is a hexadecimal digit. - Note that the Xuc property does not match these sequences but the char- - acters that they represent.) - - Resetting the match start - - The escape sequence \K causes any previously matched characters not to - be included in the final matched sequence. For example, the pattern: - - foo\Kbar - - matches "foobar", but reports that it has matched "bar". This feature - is similar to a lookbehind assertion (described below). However, in - this case, the part of the subject before the real match does not have - to be of fixed length, as lookbehind assertions do. The use of \K does - not interfere with the setting of captured substrings. For example, - when the pattern - - (foo)\Kbar - - matches "foobar", the first substring is still set to "foo". - - Perl documents that the use of \K within assertions is "not well - defined". In PCRE, \K is acted upon when it occurs inside positive - assertions, but is ignored in negative assertions. Note that when a - pattern such as (?=ab\K) matches, the reported start of the match can - be greater than the end of the match. - - Simple assertions - - The final use of backslash is for certain simple assertions. An asser- - tion specifies a condition that has to be met at a particular point in - a match, without consuming any characters from the subject string. The - use of subpatterns for more complicated assertions is described below. - The backslashed assertions are: - - \b matches at a word boundary - \B matches when not at a word boundary - \A matches at the start of the subject - \Z matches at the end of the subject - also matches before a newline at the end of the subject - \z matches only at the end of the subject - \G matches at the first matching position in the subject - - Inside a character class, \b has a different meaning; it matches the - backspace character. If any other of these assertions appears in a - character class, by default it matches the corresponding literal char- - acter (for example, \B matches the letter B). However, if the - PCRE_EXTRA option is set, an "invalid escape sequence" error is gener- - ated instead. - - A word boundary is a position in the subject string where the current - character and the previous character do not both match \w or \W (i.e. - one matches \w and the other matches \W), or the start or end of the - string if the first or last character matches \w, respectively. In a - UTF mode, the meanings of \w and \W can be changed by setting the - PCRE_UCP option. When this is done, it also affects \b and \B. Neither - PCRE nor Perl has a separate "start of word" or "end of word" metase- - quence. However, whatever follows \b normally determines which it is. - For example, the fragment \ba matches "a" at the start of a word. - - The \A, \Z, and \z assertions differ from the traditional circumflex - and dollar (described in the next section) in that they only ever match - at the very start and end of the subject string, whatever options are - set. Thus, they are independent of multiline mode. These three asser- - tions are not affected by the PCRE_NOTBOL or PCRE_NOTEOL options, which - affect only the behaviour of the circumflex and dollar metacharacters. - However, if the startoffset argument of pcre_exec() is non-zero, indi- - cating that matching is to start at a point other than the beginning of - the subject, \A can never match. The difference between \Z and \z is - that \Z matches before a newline at the end of the string as well as at - the very end, whereas \z matches only at the end. - - The \G assertion is true only when the current matching position is at - the start point of the match, as specified by the startoffset argument - of pcre_exec(). It differs from \A when the value of startoffset is - non-zero. By calling pcre_exec() multiple times with appropriate argu- - ments, you can mimic Perl's /g option, and it is in this kind of imple- - mentation where \G can be useful. - - Note, however, that PCRE's interpretation of \G, as the start of the - current match, is subtly different from Perl's, which defines it as the - end of the previous match. In Perl, these can be different when the - previously matched string was empty. Because PCRE does just one match - at a time, it cannot reproduce this behaviour. - - If all the alternatives of a pattern begin with \G, the expression is - anchored to the starting match position, and the "anchored" flag is set - in the compiled regular expression. - - -CIRCUMFLEX AND DOLLAR - - The circumflex and dollar metacharacters are zero-width assertions. - That is, they test for a particular condition being true without con- - suming any characters from the subject string. - - Outside a character class, in the default matching mode, the circumflex - character is an assertion that is true only if the current matching - point is at the start of the subject string. If the startoffset argu- - ment of pcre_exec() is non-zero, circumflex can never match if the - PCRE_MULTILINE option is unset. Inside a character class, circumflex - has an entirely different meaning (see below). - - Circumflex need not be the first character of the pattern if a number - of alternatives are involved, but it should be the first thing in each - alternative in which it appears if the pattern is ever to match that - branch. If all possible alternatives start with a circumflex, that is, - if the pattern is constrained to match only at the start of the sub- - ject, it is said to be an "anchored" pattern. (There are also other - constructs that can cause a pattern to be anchored.) - - The dollar character is an assertion that is true only if the current - matching point is at the end of the subject string, or immediately - before a newline at the end of the string (by default). Note, however, - that it does not actually match the newline. Dollar need not be the - last character of the pattern if a number of alternatives are involved, - but it should be the last item in any branch in which it appears. Dol- - lar has no special meaning in a character class. - - The meaning of dollar can be changed so that it matches only at the - very end of the string, by setting the PCRE_DOLLAR_ENDONLY option at - compile time. This does not affect the \Z assertion. - - The meanings of the circumflex and dollar characters are changed if the - PCRE_MULTILINE option is set. When this is the case, a circumflex - matches immediately after internal newlines as well as at the start of - the subject string. It does not match after a newline that ends the - string. A dollar matches before any newlines in the string, as well as - at the very end, when PCRE_MULTILINE is set. When newline is specified - as the two-character sequence CRLF, isolated CR and LF characters do - not indicate newlines. - - For example, the pattern /^abc$/ matches the subject string "def\nabc" - (where \n represents a newline) in multiline mode, but not otherwise. - Consequently, patterns that are anchored in single line mode because - all branches start with ^ are not anchored in multiline mode, and a - match for circumflex is possible when the startoffset argument of - pcre_exec() is non-zero. The PCRE_DOLLAR_ENDONLY option is ignored if - PCRE_MULTILINE is set. - - Note that the sequences \A, \Z, and \z can be used to match the start - and end of the subject in both modes, and if all branches of a pattern - start with \A it is always anchored, whether or not PCRE_MULTILINE is - set. - - -FULL STOP (PERIOD, DOT) AND \N - - Outside a character class, a dot in the pattern matches any one charac- - ter in the subject string except (by default) a character that signi- - fies the end of a line. - - When a line ending is defined as a single character, dot never matches - that character; when the two-character sequence CRLF is used, dot does - not match CR if it is immediately followed by LF, but otherwise it - matches all characters (including isolated CRs and LFs). When any Uni- - code line endings are being recognized, dot does not match CR or LF or - any of the other line ending characters. - - The behaviour of dot with regard to newlines can be changed. If the - PCRE_DOTALL option is set, a dot matches any one character, without - exception. If the two-character sequence CRLF is present in the subject - string, it takes two dots to match it. - - The handling of dot is entirely independent of the handling of circum- - flex and dollar, the only relationship being that they both involve - newlines. Dot has no special meaning in a character class. - - The escape sequence \N behaves like a dot, except that it is not - affected by the PCRE_DOTALL option. In other words, it matches any - character except one that signifies the end of a line. Perl also uses - \N to match characters by name; PCRE does not support this. - - -MATCHING A SINGLE DATA UNIT - - Outside a character class, the escape sequence \C matches any one data - unit, whether or not a UTF mode is set. In the 8-bit library, one data - unit is one byte; in the 16-bit library it is a 16-bit unit; in the - 32-bit library it is a 32-bit unit. Unlike a dot, \C always matches - line-ending characters. The feature is provided in Perl in order to - match individual bytes in UTF-8 mode, but it is unclear how it can use- - fully be used. Because \C breaks up characters into individual data - units, matching one unit with \C in a UTF mode means that the rest of - the string may start with a malformed UTF character. This has undefined - results, because PCRE assumes that it is dealing with valid UTF strings - (and by default it checks this at the start of processing unless the - PCRE_NO_UTF8_CHECK, PCRE_NO_UTF16_CHECK or PCRE_NO_UTF32_CHECK option - is used). - - PCRE does not allow \C to appear in lookbehind assertions (described - below) in a UTF mode, because this would make it impossible to calcu- - late the length of the lookbehind. - - In general, the \C escape sequence is best avoided. However, one way of - using it that avoids the problem of malformed UTF characters is to use - a lookahead to check the length of the next character, as in this pat- - tern, which could be used with a UTF-8 string (ignore white space and - line breaks): - - (?| (?=[\x00-\x7f])(\C) | - (?=[\x80-\x{7ff}])(\C)(\C) | - (?=[\x{800}-\x{ffff}])(\C)(\C)(\C) | - (?=[\x{10000}-\x{1fffff}])(\C)(\C)(\C)(\C)) - - A group that starts with (?| resets the capturing parentheses numbers - in each alternative (see "Duplicate Subpattern Numbers" below). The - assertions at the start of each branch check the next UTF-8 character - for values whose encoding uses 1, 2, 3, or 4 bytes, respectively. The - character's individual bytes are then captured by the appropriate num- - ber of groups. - - -SQUARE BRACKETS AND CHARACTER CLASSES - - An opening square bracket introduces a character class, terminated by a - closing square bracket. A closing square bracket on its own is not spe- - cial by default. However, if the PCRE_JAVASCRIPT_COMPAT option is set, - a lone closing square bracket causes a compile-time error. If a closing - square bracket is required as a member of the class, it should be the - first data character in the class (after an initial circumflex, if - present) or escaped with a backslash. - - A character class matches a single character in the subject. In a UTF - mode, the character may be more than one data unit long. A matched - character must be in the set of characters defined by the class, unless - the first character in the class definition is a circumflex, in which - case the subject character must not be in the set defined by the class. - If a circumflex is actually required as a member of the class, ensure - it is not the first character, or escape it with a backslash. - - For example, the character class [aeiou] matches any lower case vowel, - while [^aeiou] matches any character that is not a lower case vowel. - Note that a circumflex is just a convenient notation for specifying the - characters that are in the class by enumerating those that are not. A - class that starts with a circumflex is not an assertion; it still con- - sumes a character from the subject string, and therefore it fails if - the current pointer is at the end of the string. - - In UTF-8 (UTF-16, UTF-32) mode, characters with values greater than 255 - (0xffff) can be included in a class as a literal string of data units, - or by using the \x{ escaping mechanism. - - When caseless matching is set, any letters in a class represent both - their upper case and lower case versions, so for example, a caseless - [aeiou] matches "A" as well as "a", and a caseless [^aeiou] does not - match "A", whereas a caseful version would. In a UTF mode, PCRE always - understands the concept of case for characters whose values are less - than 128, so caseless matching is always possible. For characters with - higher values, the concept of case is supported if PCRE is compiled - with Unicode property support, but not otherwise. If you want to use - caseless matching in a UTF mode for characters 128 and above, you must - ensure that PCRE is compiled with Unicode property support as well as - with UTF support. - - Characters that might indicate line breaks are never treated in any - special way when matching character classes, whatever line-ending - sequence is in use, and whatever setting of the PCRE_DOTALL and - PCRE_MULTILINE options is used. A class such as [^a] always matches one - of these characters. - - The minus (hyphen) character can be used to specify a range of charac- - ters in a character class. For example, [d-m] matches any letter - between d and m, inclusive. If a minus character is required in a - class, it must be escaped with a backslash or appear in a position - where it cannot be interpreted as indicating a range, typically as the - first or last character in the class, or immediately after a range. For - example, [b-d-z] matches letters in the range b to d, a hyphen charac- - ter, or z. - - It is not possible to have the literal character "]" as the end charac- - ter of a range. A pattern such as [W-]46] is interpreted as a class of - two characters ("W" and "-") followed by a literal string "46]", so it - would match "W46]" or "-46]". However, if the "]" is escaped with a - backslash it is interpreted as the end of range, so [W-\]46] is inter- - preted as a class containing a range followed by two other characters. - The octal or hexadecimal representation of "]" can also be used to end - a range. - - An error is generated if a POSIX character class (see below) or an - escape sequence other than one that defines a single character appears - at a point where a range ending character is expected. For example, - [z-\xff] is valid, but [A-\d] and [A-[:digit:]] are not. - - Ranges operate in the collating sequence of character values. They can - also be used for characters specified numerically, for example - [\000-\037]. Ranges can include any characters that are valid for the - current mode. - - If a range that includes letters is used when caseless matching is set, - it matches the letters in either case. For example, [W-c] is equivalent - to [][\\^_`wxyzabc], matched caselessly, and in a non-UTF mode, if - character tables for a French locale are in use, [\xc8-\xcb] matches - accented E characters in both cases. In UTF modes, PCRE supports the - concept of case for characters with values greater than 128 only when - it is compiled with Unicode property support. - - The character escape sequences \d, \D, \h, \H, \p, \P, \s, \S, \v, \V, - \w, and \W may appear in a character class, and add the characters that - they match to the class. For example, [\dABCDEF] matches any hexadeci- - mal digit. In UTF modes, the PCRE_UCP option affects the meanings of - \d, \s, \w and their upper case partners, just as it does when they - appear outside a character class, as described in the section entitled - "Generic character types" above. The escape sequence \b has a different - meaning inside a character class; it matches the backspace character. - The sequences \B, \N, \R, and \X are not special inside a character - class. Like any other unrecognized escape sequences, they are treated - as the literal characters "B", "N", "R", and "X" by default, but cause - an error if the PCRE_EXTRA option is set. - - A circumflex can conveniently be used with the upper case character - types to specify a more restricted set of characters than the matching - lower case type. For example, the class [^\W_] matches any letter or - digit, but not underscore, whereas [\w] includes underscore. A positive - character class should be read as "something OR something OR ..." and a - negative class as "NOT something AND NOT something AND NOT ...". - - The only metacharacters that are recognized in character classes are - backslash, hyphen (only where it can be interpreted as specifying a - range), circumflex (only at the start), opening square bracket (only - when it can be interpreted as introducing a POSIX class name, or for a - special compatibility feature - see the next two sections), and the - terminating closing square bracket. However, escaping other non- - alphanumeric characters does no harm. - - -POSIX CHARACTER CLASSES - - Perl supports the POSIX notation for character classes. This uses names - enclosed by [: and :] within the enclosing square brackets. PCRE also - supports this notation. For example, - - [01[:alpha:]%] - - matches "0", "1", any alphabetic character, or "%". The supported class - names are: - - alnum letters and digits - alpha letters - ascii character codes 0 - 127 - blank space or tab only - cntrl control characters - digit decimal digits (same as \d) - graph printing characters, excluding space - lower lower case letters - print printing characters, including space - punct printing characters, excluding letters and digits and space - space white space (the same as \s from PCRE 8.34) - upper upper case letters - word "word" characters (same as \w) - xdigit hexadecimal digits - - The default "space" characters are HT (9), LF (10), VT (11), FF (12), - CR (13), and space (32). If locale-specific matching is taking place, - the list of space characters may be different; there may be fewer or - more of them. "Space" used to be different to \s, which did not include - VT, for Perl compatibility. However, Perl changed at release 5.18, and - PCRE followed at release 8.34. "Space" and \s now match the same set - of characters. - - The name "word" is a Perl extension, and "blank" is a GNU extension - from Perl 5.8. Another Perl extension is negation, which is indicated - by a ^ character after the colon. For example, - - [12[:^digit:]] - - matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the - POSIX syntax [.ch.] and [=ch=] where "ch" is a "collating element", but - these are not supported, and an error is given if they are encountered. - - By default, characters with values greater than 128 do not match any of - the POSIX character classes. However, if the PCRE_UCP option is passed - to pcre_compile(), some of the classes are changed so that Unicode - character properties are used. This is achieved by replacing certain - POSIX classes by other sequences, as follows: - - [:alnum:] becomes \p{Xan} - [:alpha:] becomes \p{L} - [:blank:] becomes \h - [:digit:] becomes \p{Nd} - [:lower:] becomes \p{Ll} - [:space:] becomes \p{Xps} - [:upper:] becomes \p{Lu} - [:word:] becomes \p{Xwd} - - Negated versions, such as [:^alpha:] use \P instead of \p. Three other - POSIX classes are handled specially in UCP mode: - - [:graph:] This matches characters that have glyphs that mark the page - when printed. In Unicode property terms, it matches all char- - acters with the L, M, N, P, S, or Cf properties, except for: - - U+061C Arabic Letter Mark - U+180E Mongolian Vowel Separator - U+2066 - U+2069 Various "isolate"s - - - [:print:] This matches the same characters as [:graph:] plus space - characters that are not controls, that is, characters with - the Zs property. - - [:punct:] This matches all characters that have the Unicode P (punctua- - tion) property, plus those characters whose code points are - less than 128 that have the S (Symbol) property. - - The other POSIX classes are unchanged, and match only characters with - code points less than 128. - - -COMPATIBILITY FEATURE FOR WORD BOUNDARIES - - In the POSIX.2 compliant library that was included in 4.4BSD Unix, the - ugly syntax [[:<:]] and [[:>:]] is used for matching "start of word" - and "end of word". PCRE treats these items as follows: - - [[:<:]] is converted to \b(?=\w) - [[:>:]] is converted to \b(?<=\w) - - Only these exact character sequences are recognized. A sequence such as - [a[:<:]b] provokes error for an unrecognized POSIX class name. This - support is not compatible with Perl. It is provided to help migrations - from other environments, and is best not used in any new patterns. Note - that \b matches at the start and the end of a word (see "Simple asser- - tions" above), and in a Perl-style pattern the preceding or following - character normally shows which is wanted, without the need for the - assertions that are used above in order to give exactly the POSIX be- - haviour. - - -VERTICAL BAR - - Vertical bar characters are used to separate alternative patterns. For - example, the pattern - - gilbert|sullivan - - matches either "gilbert" or "sullivan". Any number of alternatives may - appear, and an empty alternative is permitted (matching the empty - string). The matching process tries each alternative in turn, from left - to right, and the first one that succeeds is used. If the alternatives - are within a subpattern (defined below), "succeeds" means matching the - rest of the main pattern as well as the alternative in the subpattern. - - -INTERNAL OPTION SETTING - - The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and - PCRE_EXTENDED options (which are Perl-compatible) can be changed from - within the pattern by a sequence of Perl option letters enclosed - between "(?" and ")". The option letters are - - i for PCRE_CASELESS - m for PCRE_MULTILINE - s for PCRE_DOTALL - x for PCRE_EXTENDED - - For example, (?im) sets caseless, multiline matching. It is also possi- - ble to unset these options by preceding the letter with a hyphen, and a - combined setting and unsetting such as (?im-sx), which sets PCRE_CASE- - LESS and PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, - is also permitted. If a letter appears both before and after the - hyphen, the option is unset. - - The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA - can be changed in the same way as the Perl-compatible options by using - the characters J, U and X respectively. - - When one of these option changes occurs at top level (that is, not - inside subpattern parentheses), the change applies to the remainder of - the pattern that follows. If the change is placed right at the start of - a pattern, PCRE extracts it into the global options (and it will there- - fore show up in data extracted by the pcre_fullinfo() function). - - An option change within a subpattern (see below for a description of - subpatterns) affects only that part of the subpattern that follows it, - so - - (a(?i)b)c - - matches abc and aBc and no other strings (assuming PCRE_CASELESS is not - used). By this means, options can be made to have different settings - in different parts of the pattern. Any changes made in one alternative - do carry on into subsequent branches within the same subpattern. For - example, - - (a(?i)b|c) - - matches "ab", "aB", "c", and "C", even though when matching "C" the - first branch is abandoned before the option setting. This is because - the effects of option settings happen at compile time. There would be - some very weird behaviour otherwise. - - Note: There are other PCRE-specific options that can be set by the - application when the compiling or matching functions are called. In - some cases the pattern can contain special leading sequences such as - (*CRLF) to override what the application has set or what has been - defaulted. Details are given in the section entitled "Newline - sequences" above. There are also the (*UTF8), (*UTF16),(*UTF32), and - (*UCP) leading sequences that can be used to set UTF and Unicode prop- - erty modes; they are equivalent to setting the PCRE_UTF8, PCRE_UTF16, - PCRE_UTF32 and the PCRE_UCP options, respectively. The (*UTF) sequence - is a generic version that can be used with any of the libraries. How- - ever, the application can set the PCRE_NEVER_UTF option, which locks - out the use of the (*UTF) sequences. - - -SUBPATTERNS - - Subpatterns are delimited by parentheses (round brackets), which can be - nested. Turning part of a pattern into a subpattern does two things: - - 1. It localizes a set of alternatives. For example, the pattern - - cat(aract|erpillar|) - - matches "cataract", "caterpillar", or "cat". Without the parentheses, - it would match "cataract", "erpillar" or an empty string. - - 2. It sets up the subpattern as a capturing subpattern. This means - that, when the whole pattern matches, that portion of the subject - string that matched the subpattern is passed back to the caller via the - ovector argument of the matching function. (This applies only to the - traditional matching functions; the DFA matching functions do not sup- - port capturing.) - - Opening parentheses are counted from left to right (starting from 1) to - obtain numbers for the capturing subpatterns. For example, if the - string "the red king" is matched against the pattern - - the ((red|white) (king|queen)) - - the captured substrings are "red king", "red", and "king", and are num- - bered 1, 2, and 3, respectively. - - The fact that plain parentheses fulfil two functions is not always - helpful. There are often times when a grouping subpattern is required - without a capturing requirement. If an opening parenthesis is followed - by a question mark and a colon, the subpattern does not do any captur- - ing, and is not counted when computing the number of any subsequent - capturing subpatterns. For example, if the string "the white queen" is - matched against the pattern - - the ((?:red|white) (king|queen)) - - the captured substrings are "white queen" and "queen", and are numbered - 1 and 2. The maximum number of capturing subpatterns is 65535. - - As a convenient shorthand, if any option settings are required at the - start of a non-capturing subpattern, the option letters may appear - between the "?" and the ":". Thus the two patterns - - (?i:saturday|sunday) - (?:(?i)saturday|sunday) - - match exactly the same set of strings. Because alternative branches are - tried from left to right, and options are not reset until the end of - the subpattern is reached, an option setting in one branch does affect - subsequent branches, so the above patterns match "SUNDAY" as well as - "Saturday". - - -DUPLICATE SUBPATTERN NUMBERS - - Perl 5.10 introduced a feature whereby each alternative in a subpattern - uses the same numbers for its capturing parentheses. Such a subpattern - starts with (?| and is itself a non-capturing subpattern. For example, - consider this pattern: - - (?|(Sat)ur|(Sun))day - - Because the two alternatives are inside a (?| group, both sets of cap- - turing parentheses are numbered one. Thus, when the pattern matches, - you can look at captured substring number one, whichever alternative - matched. This construct is useful when you want to capture part, but - not all, of one of a number of alternatives. Inside a (?| group, paren- - theses are numbered as usual, but the number is reset at the start of - each branch. The numbers of any capturing parentheses that follow the - subpattern start after the highest number used in any branch. The fol- - lowing example is taken from the Perl documentation. The numbers under- - neath show in which buffer the captured content will be stored. - - # before ---------------branch-reset----------- after - / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x - # 1 2 2 3 2 3 4 - - A back reference to a numbered subpattern uses the most recent value - that is set for that number by any subpattern. The following pattern - matches "abcabc" or "defdef": - - /(?|(abc)|(def))\1/ - - In contrast, a subroutine call to a numbered subpattern always refers - to the first one in the pattern with the given number. The following - pattern matches "abcabc" or "defabc": - - /(?|(abc)|(def))(?1)/ - - If a condition test for a subpattern's having matched refers to a non- - unique number, the test is true if any of the subpatterns of that num- - ber have matched. - - An alternative approach to using this "branch reset" feature is to use - duplicate named subpatterns, as described in the next section. - - -NAMED SUBPATTERNS - - Identifying capturing parentheses by number is simple, but it can be - very hard to keep track of the numbers in complicated regular expres- - sions. Furthermore, if an expression is modified, the numbers may - change. To help with this difficulty, PCRE supports the naming of sub- - patterns. This feature was not added to Perl until release 5.10. Python - had the feature earlier, and PCRE introduced it at release 4.0, using - the Python syntax. PCRE now supports both the Perl and the Python syn- - tax. Perl allows identically numbered subpatterns to have different - names, but PCRE does not. - - In PCRE, a subpattern can be named in one of three ways: (?<name>...) - or (?'name'...) as in Perl, or (?P<name>...) as in Python. References - to capturing parentheses from other parts of the pattern, such as back - references, recursion, and conditions, can be made by name as well as - by number. - - Names consist of up to 32 alphanumeric characters and underscores, but - must start with a non-digit. Named capturing parentheses are still - allocated numbers as well as names, exactly as if the names were not - present. The PCRE API provides function calls for extracting the name- - to-number translation table from a compiled pattern. There is also a - convenience function for extracting a captured substring by name. - - By default, a name must be unique within a pattern, but it is possible - to relax this constraint by setting the PCRE_DUPNAMES option at compile - time. (Duplicate names are also always permitted for subpatterns with - the same number, set up as described in the previous section.) Dupli- - cate names can be useful for patterns where only one instance of the - named parentheses can match. Suppose you want to match the name of a - weekday, either as a 3-letter abbreviation or as the full name, and in - both cases you want to extract the abbreviation. This pattern (ignoring - the line breaks) does the job: - - (?<DN>Mon|Fri|Sun)(?:day)?| - (?<DN>Tue)(?:sday)?| - (?<DN>Wed)(?:nesday)?| - (?<DN>Thu)(?:rsday)?| - (?<DN>Sat)(?:urday)? - - There are five capturing substrings, but only one is ever set after a - match. (An alternative way of solving this problem is to use a "branch - reset" subpattern, as described in the previous section.) - - The convenience function for extracting the data by name returns the - substring for the first (and in this example, the only) subpattern of - that name that matched. This saves searching to find which numbered - subpattern it was. - - If you make a back reference to a non-unique named subpattern from - elsewhere in the pattern, the subpatterns to which the name refers are - checked in the order in which they appear in the overall pattern. The - first one that is set is used for the reference. For example, this pat- - tern matches both "foofoo" and "barbar" but not "foobar" or "barfoo": - - (?:(?<n>foo)|(?<n>bar))\k<n> - - - If you make a subroutine call to a non-unique named subpattern, the one - that corresponds to the first occurrence of the name is used. In the - absence of duplicate numbers (see the previous section) this is the one - with the lowest number. - - If you use a named reference in a condition test (see the section about - conditions below), either to check whether a subpattern has matched, or - to check for recursion, all subpatterns with the same name are tested. - If the condition is true for any one of them, the overall condition is - true. This is the same behaviour as testing by number. For further - details of the interfaces for handling named subpatterns, see the - pcreapi documentation. - - Warning: You cannot use different names to distinguish between two sub- - patterns with the same number because PCRE uses only the numbers when - matching. For this reason, an error is given at compile time if differ- - ent names are given to subpatterns with the same number. However, you - can always give the same name to subpatterns with the same number, even - when PCRE_DUPNAMES is not set. - - -REPETITION - - Repetition is specified by quantifiers, which can follow any of the - following items: - - a literal data character - the dot metacharacter - the \C escape sequence - the \X escape sequence - the \R escape sequence - an escape such as \d or \pL that matches a single character - a character class - a back reference (see next section) - a parenthesized subpattern (including assertions) - a subroutine call to a subpattern (recursive or otherwise) - - The general repetition quantifier specifies a minimum and maximum num- - ber of permitted matches, by giving the two numbers in curly brackets - (braces), separated by a comma. The numbers must be less than 65536, - and the first must be less than or equal to the second. For example: - - z{2,4} - - matches "zz", "zzz", or "zzzz". A closing brace on its own is not a - special character. If the second number is omitted, but the comma is - present, there is no upper limit; if the second number and the comma - are both omitted, the quantifier specifies an exact number of required - matches. Thus - - [aeiou]{3,} - - matches at least 3 successive vowels, but may match many more, while - - \d{8} - - matches exactly 8 digits. An opening curly bracket that appears in a - position where a quantifier is not allowed, or one that does not match - the syntax of a quantifier, is taken as a literal character. For exam- - ple, {,6} is not a quantifier, but a literal string of four characters. - - In UTF modes, quantifiers apply to characters rather than to individual - data units. Thus, for example, \x{100}{2} matches two characters, each - of which is represented by a two-byte sequence in a UTF-8 string. Simi- - larly, \X{3} matches three Unicode extended grapheme clusters, each of - which may be several data units long (and they may be of different - lengths). - - The quantifier {0} is permitted, causing the expression to behave as if - the previous item and the quantifier were not present. This may be use- - ful for subpatterns that are referenced as subroutines from elsewhere - in the pattern (but see also the section entitled "Defining subpatterns - for use by reference only" below). Items other than subpatterns that - have a {0} quantifier are omitted from the compiled pattern. - - For convenience, the three most common quantifiers have single-charac- - ter abbreviations: - - * is equivalent to {0,} - + is equivalent to {1,} - ? is equivalent to {0,1} - - It is possible to construct infinite loops by following a subpattern - that can match no characters with a quantifier that has no upper limit, - for example: - - (a?)* - - Earlier versions of Perl and PCRE used to give an error at compile time - for such patterns. However, because there are cases where this can be - useful, such patterns are now accepted, but if any repetition of the - subpattern does in fact match no characters, the loop is forcibly bro- - ken. - - By default, the quantifiers are "greedy", that is, they match as much - as possible (up to the maximum number of permitted times), without - causing the rest of the pattern to fail. The classic example of where - this gives problems is in trying to match comments in C programs. These - appear between /* and */ and within the comment, individual * and / - characters may appear. An attempt to match C comments by applying the - pattern - - /\*.*\*/ - - to the string - - /* first comment */ not comment /* second comment */ - - fails, because it matches the entire string owing to the greediness of - the .* item. - - However, if a quantifier is followed by a question mark, it ceases to - be greedy, and instead matches the minimum number of times possible, so - the pattern - - /\*.*?\*/ - - does the right thing with the C comments. The meaning of the various - quantifiers is not otherwise changed, just the preferred number of - matches. Do not confuse this use of question mark with its use as a - quantifier in its own right. Because it has two uses, it can sometimes - appear doubled, as in - - \d??\d - - which matches one digit by preference, but can match two if that is the - only way the rest of the pattern matches. - - If the PCRE_UNGREEDY option is set (an option that is not available in - Perl), the quantifiers are not greedy by default, but individual ones - can be made greedy by following them with a question mark. In other - words, it inverts the default behaviour. - - When a parenthesized subpattern is quantified with a minimum repeat - count that is greater than 1 or with a limited maximum, more memory is - required for the compiled pattern, in proportion to the size of the - minimum or maximum. - - If a pattern starts with .* or .{0,} and the PCRE_DOTALL option (equiv- - alent to Perl's /s) is set, thus allowing the dot to match newlines, - the pattern is implicitly anchored, because whatever follows will be - tried against every character position in the subject string, so there - is no point in retrying the overall match at any position after the - first. PCRE normally treats such a pattern as though it were preceded - by \A. - - In cases where it is known that the subject string contains no new- - lines, it is worth setting PCRE_DOTALL in order to obtain this opti- - mization, or alternatively using ^ to indicate anchoring explicitly. - - However, there are some cases where the optimization cannot be used. - When .* is inside capturing parentheses that are the subject of a back - reference elsewhere in the pattern, a match at the start may fail where - a later one succeeds. Consider, for example: - - (.*)abc\1 - - If the subject is "xyz123abc123" the match point is the fourth charac- - ter. For this reason, such a pattern is not implicitly anchored. - - Another case where implicit anchoring is not applied is when the lead- - ing .* is inside an atomic group. Once again, a match at the start may - fail where a later one succeeds. Consider this pattern: - - (?>.*?a)b - - It matches "ab" in the subject "aab". The use of the backtracking con- - trol verbs (*PRUNE) and (*SKIP) also disable this optimization. - - When a capturing subpattern is repeated, the value captured is the sub- - string that matched the final iteration. For example, after - - (tweedle[dume]{3}\s*)+ - - has matched "tweedledum tweedledee" the value of the captured substring - is "tweedledee". However, if there are nested capturing subpatterns, - the corresponding captured values may have been set in previous itera- - tions. For example, after - - /(a|(b))+/ - - matches "aba" the value of the second captured substring is "b". - - -ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS - - With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") - repetition, failure of what follows normally causes the repeated item - to be re-evaluated to see if a different number of repeats allows the - rest of the pattern to match. Sometimes it is useful to prevent this, - either to change the nature of the match, or to cause it fail earlier - than it otherwise might, when the author of the pattern knows there is - no point in carrying on. - - Consider, for example, the pattern \d+foo when applied to the subject - line - - 123456bar - - After matching all 6 digits and then failing to match "foo", the normal - action of the matcher is to try again with only 5 digits matching the - \d+ item, and then with 4, and so on, before ultimately failing. - "Atomic grouping" (a term taken from Jeffrey Friedl's book) provides - the means for specifying that once a subpattern has matched, it is not - to be re-evaluated in this way. - - If we use atomic grouping for the previous example, the matcher gives - up immediately on failing to match "foo" the first time. The notation - is a kind of special parenthesis, starting with (?> as in this example: - - (?>\d+)foo - - This kind of parenthesis "locks up" the part of the pattern it con- - tains once it has matched, and a failure further into the pattern is - prevented from backtracking into it. Backtracking past it to previous - items, however, works as normal. - - An alternative description is that a subpattern of this type matches - the string of characters that an identical standalone pattern would - match, if anchored at the current point in the subject string. - - Atomic grouping subpatterns are not capturing subpatterns. Simple cases - such as the above example can be thought of as a maximizing repeat that - must swallow everything it can. So, while both \d+ and \d+? are pre- - pared to adjust the number of digits they match in order to make the - rest of the pattern match, (?>\d+) can only match an entire sequence of - digits. - - Atomic groups in general can of course contain arbitrarily complicated - subpatterns, and can be nested. However, when the subpattern for an - atomic group is just a single repeated item, as in the example above, a - simpler notation, called a "possessive quantifier" can be used. This - consists of an additional + character following a quantifier. Using - this notation, the previous example can be rewritten as - - \d++foo - - Note that a possessive quantifier can be used with an entire group, for - example: - - (abc|xyz){2,3}+ - - Possessive quantifiers are always greedy; the setting of the - PCRE_UNGREEDY option is ignored. They are a convenient notation for the - simpler forms of atomic group. However, there is no difference in the - meaning of a possessive quantifier and the equivalent atomic group, - though there may be a performance difference; possessive quantifiers - should be slightly faster. - - The possessive quantifier syntax is an extension to the Perl 5.8 syn- - tax. Jeffrey Friedl originated the idea (and the name) in the first - edition of his book. Mike McCloskey liked it, so implemented it when he - built Sun's Java package, and PCRE copied it from there. It ultimately - found its way into Perl at release 5.10. - - PCRE has an optimization that automatically "possessifies" certain sim- - ple pattern constructs. For example, the sequence A+B is treated as - A++B because there is no point in backtracking into a sequence of A's - when B must follow. - - When a pattern contains an unlimited repeat inside a subpattern that - can itself be repeated an unlimited number of times, the use of an - atomic group is the only way to avoid some failing matches taking a - very long time indeed. The pattern - - (\D+|<\d+>)*[!?] - - matches an unlimited number of substrings that either consist of non- - digits, or digits enclosed in <>, followed by either ! or ?. When it - matches, it runs quickly. However, if it is applied to - - aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa - - it takes a long time before reporting failure. This is because the - string can be divided between the internal \D+ repeat and the external - * repeat in a large number of ways, and all have to be tried. (The - example uses [!?] rather than a single character at the end, because - both PCRE and Perl have an optimization that allows for fast failure - when a single character is used. They remember the last single charac- - ter that is required for a match, and fail early if it is not present - in the string.) If the pattern is changed so that it uses an atomic - group, like this: - - ((?>\D+)|<\d+>)*[!?] - - sequences of non-digits cannot be broken, and failure happens quickly. - - -BACK REFERENCES - - Outside a character class, a backslash followed by a digit greater than - 0 (and possibly further digits) is a back reference to a capturing sub- - pattern earlier (that is, to its left) in the pattern, provided there - have been that many previous capturing left parentheses. - - However, if the decimal number following the backslash is less than 10, - it is always taken as a back reference, and causes an error only if - there are not that many capturing left parentheses in the entire pat- - tern. In other words, the parentheses that are referenced need not be - to the left of the reference for numbers less than 10. A "forward back - reference" of this type can make sense when a repetition is involved - and the subpattern to the right has participated in an earlier itera- - tion. - - It is not possible to have a numerical "forward back reference" to a - subpattern whose number is 10 or more using this syntax because a - sequence such as \50 is interpreted as a character defined in octal. - See the subsection entitled "Non-printing characters" above for further - details of the handling of digits following a backslash. There is no - such problem when named parentheses are used. A back reference to any - subpattern is possible using named parentheses (see below). - - Another way of avoiding the ambiguity inherent in the use of digits - following a backslash is to use the \g escape sequence. This escape - must be followed by an unsigned number or a negative number, optionally - enclosed in braces. These examples are all identical: - - (ring), \1 - (ring), \g1 - (ring), \g{1} - - An unsigned number specifies an absolute reference without the ambigu- - ity that is present in the older syntax. It is also useful when literal - digits follow the reference. A negative number is a relative reference. - Consider this example: - - (abc(def)ghi)\g{-1} - - The sequence \g{-1} is a reference to the most recently started captur- - ing subpattern before \g, that is, is it equivalent to \2 in this exam- - ple. Similarly, \g{-2} would be equivalent to \1. The use of relative - references can be helpful in long patterns, and also in patterns that - are created by joining together fragments that contain references - within themselves. - - A back reference matches whatever actually matched the capturing sub- - pattern in the current subject string, rather than anything matching - the subpattern itself (see "Subpatterns as subroutines" below for a way - of doing that). So the pattern - - (sens|respons)e and \1ibility - - matches "sense and sensibility" and "response and responsibility", but - not "sense and responsibility". If caseful matching is in force at the - time of the back reference, the case of letters is relevant. For exam- - ple, - - ((?i)rah)\s+\1 - - matches "rah rah" and "RAH RAH", but not "RAH rah", even though the - original capturing subpattern is matched caselessly. - - There are several different ways of writing back references to named - subpatterns. The .NET syntax \k{name} and the Perl syntax \k<name> or - \k'name' are supported, as is the Python syntax (?P=name). Perl 5.10's - unified back reference syntax, in which \g can be used for both numeric - and named references, is also supported. We could rewrite the above - example in any of the following ways: - - (?<p1>(?i)rah)\s+\k<p1> - (?'p1'(?i)rah)\s+\k{p1} - (?P<p1>(?i)rah)\s+(?P=p1) - (?<p1>(?i)rah)\s+\g{p1} - - A subpattern that is referenced by name may appear in the pattern - before or after the reference. - - There may be more than one back reference to the same subpattern. If a - subpattern has not actually been used in a particular match, any back - references to it always fail by default. For example, the pattern - - (a|(bc))\2 - - always fails if it starts to match "a" rather than "bc". However, if - the PCRE_JAVASCRIPT_COMPAT option is set at compile time, a back refer- - ence to an unset value matches an empty string. - - Because there may be many capturing parentheses in a pattern, all dig- - its following a backslash are taken as part of a potential back refer- - ence number. If the pattern continues with a digit character, some - delimiter must be used to terminate the back reference. If the - PCRE_EXTENDED option is set, this can be white space. Otherwise, the - \g{ syntax or an empty comment (see "Comments" below) can be used. - - Recursive back references - - A back reference that occurs inside the parentheses to which it refers - fails when the subpattern is first used, so, for example, (a\1) never - matches. However, such references can be useful inside repeated sub- - patterns. For example, the pattern - - (a|b\1)+ - - matches any number of "a"s and also "aba", "ababbaa" etc. At each iter- - ation of the subpattern, the back reference matches the character - string corresponding to the previous iteration. In order for this to - work, the pattern must be such that the first iteration does not need - to match the back reference. This can be done using alternation, as in - the example above, or by a quantifier with a minimum of zero. - - Back references of this type cause the group that they reference to be - treated as an atomic group. Once the whole group has been matched, a - subsequent matching failure cannot cause backtracking into the middle - of the group. - - -ASSERTIONS - - An assertion is a test on the characters following or preceding the - current matching point that does not actually consume any characters. - The simple assertions coded as \b, \B, \A, \G, \Z, \z, ^ and $ are - described above. - - More complicated assertions are coded as subpatterns. There are two - kinds: those that look ahead of the current position in the subject - string, and those that look behind it. An assertion subpattern is - matched in the normal way, except that it does not cause the current - matching position to be changed. - - Assertion subpatterns are not capturing subpatterns. If such an asser- - tion contains capturing subpatterns within it, these are counted for - the purposes of numbering the capturing subpatterns in the whole pat- - tern. However, substring capturing is carried out only for positive - assertions. (Perl sometimes, but not always, does do capturing in nega- - tive assertions.) - - For compatibility with Perl, assertion subpatterns may be repeated; - though it makes no sense to assert the same thing several times, the - side effect of capturing parentheses may occasionally be useful. In - practice, there only three cases: - - (1) If the quantifier is {0}, the assertion is never obeyed during - matching. However, it may contain internal capturing parenthesized - groups that are called from elsewhere via the subroutine mechanism. - - (2) If quantifier is {0,n} where n is greater than zero, it is treated - as if it were {0,1}. At run time, the rest of the pattern match is - tried with and without the assertion, the order depending on the greed- - iness of the quantifier. - - (3) If the minimum repetition is greater than zero, the quantifier is - ignored. The assertion is obeyed just once when encountered during - matching. - - Lookahead assertions - - Lookahead assertions start with (?= for positive assertions and (?! for - negative assertions. For example, - - \w+(?=;) - - matches a word followed by a semicolon, but does not include the semi- - colon in the match, and - - foo(?!bar) - - matches any occurrence of "foo" that is not followed by "bar". Note - that the apparently similar pattern - - (?!foo)bar - - does not find an occurrence of "bar" that is preceded by something - other than "foo"; it finds any occurrence of "bar" whatsoever, because - the assertion (?!foo) is always true when the next three characters are - "bar". A lookbehind assertion is needed to achieve the other effect. - - If you want to force a matching failure at some point in a pattern, the - most convenient way to do it is with (?!) because an empty string - always matches, so an assertion that requires there not to be an empty - string must always fail. The backtracking control verb (*FAIL) or (*F) - is a synonym for (?!). - - Lookbehind assertions - - Lookbehind assertions start with (?<= for positive assertions and (?<! - for negative assertions. For example, - - (?<!foo)bar - - does find an occurrence of "bar" that is not preceded by "foo". The - contents of a lookbehind assertion are restricted such that all the - strings it matches must have a fixed length. However, if there are sev- - eral top-level alternatives, they do not all have to have the same - fixed length. Thus - - (?<=bullock|donkey) - - is permitted, but - - (?<!dogs?|cats?) - - causes an error at compile time. Branches that match different length - strings are permitted only at the top level of a lookbehind assertion. - This is an extension compared with Perl, which requires all branches to - match the same length of string. An assertion such as - - (?<=ab(c|de)) - - is not permitted, because its single top-level branch can match two - different lengths, but it is acceptable to PCRE if rewritten to use two - top-level branches: - - (?<=abc|abde) - - In some cases, the escape sequence \K (see above) can be used instead - of a lookbehind assertion to get round the fixed-length restriction. - - The implementation of lookbehind assertions is, for each alternative, - to temporarily move the current position back by the fixed length and - then try to match. If there are insufficient characters before the cur- - rent position, the assertion fails. - - In a UTF mode, PCRE does not allow the \C escape (which matches a sin- - gle data unit even in a UTF mode) to appear in lookbehind assertions, - because it makes it impossible to calculate the length of the lookbe- - hind. The \X and \R escapes, which can match different numbers of data - units, are also not permitted. - - "Subroutine" calls (see below) such as (?2) or (?&X) are permitted in - lookbehinds, as long as the subpattern matches a fixed-length string. - Recursion, however, is not supported. - - Possessive quantifiers can be used in conjunction with lookbehind - assertions to specify efficient matching of fixed-length strings at the - end of subject strings. Consider a simple pattern such as - - abcd$ - - when applied to a long string that does not match. Because matching - proceeds from left to right, PCRE will look for each "a" in the subject - and then see if what follows matches the rest of the pattern. If the - pattern is specified as - - ^.*abcd$ - - the initial .* matches the entire string at first, but when this fails - (because there is no following "a"), it backtracks to match all but the - last character, then all but the last two characters, and so on. Once - again the search for "a" covers the entire string, from right to left, - so we are no better off. However, if the pattern is written as - - ^.*+(?<=abcd) - - there can be no backtracking for the .*+ item; it can match only the - entire string. The subsequent lookbehind assertion does a single test - on the last four characters. If it fails, the match fails immediately. - For long strings, this approach makes a significant difference to the - processing time. - - Using multiple assertions - - Several assertions (of any sort) may occur in succession. For example, - - (?<=\d{3})(?<!999)foo - - matches "foo" preceded by three digits that are not "999". Notice that - each of the assertions is applied independently at the same point in - the subject string. First there is a check that the previous three - characters are all digits, and then there is a check that the same - three characters are not "999". This pattern does not match "foo" pre- - ceded by six characters, the first of which are digits and the last - three of which are not "999". For example, it doesn't match "123abc- - foo". A pattern to do that is - - (?<=\d{3}...)(?<!999)foo - - This time the first assertion looks at the preceding six characters, - checking that the first three are digits, and then the second assertion - checks that the preceding three characters are not "999". - - Assertions can be nested in any combination. For example, - - (?<=(?<!foo)bar)baz - - matches an occurrence of "baz" that is preceded by "bar" which in turn - is not preceded by "foo", while - - (?<=\d{3}(?!999)...)foo - - is another pattern that matches "foo" preceded by three digits and any - three characters that are not "999". - - -CONDITIONAL SUBPATTERNS - - It is possible to cause the matching process to obey a subpattern con- - ditionally or to choose between two alternative subpatterns, depending - on the result of an assertion, or whether a specific capturing subpat- - tern has already been matched. The two possible forms of conditional - subpattern are: - - (?(condition)yes-pattern) - (?(condition)yes-pattern|no-pattern) - - If the condition is satisfied, the yes-pattern is used; otherwise the - no-pattern (if present) is used. If there are more than two alterna- - tives in the subpattern, a compile-time error occurs. Each of the two - alternatives may itself contain nested subpatterns of any form, includ- - ing conditional subpatterns; the restriction to two alternatives - applies only at the level of the condition. This pattern fragment is an - example where the alternatives are complex: - - (?(1) (A|B|C) | (D | (?(2)E|F) | E) ) - - - There are four kinds of condition: references to subpatterns, refer- - ences to recursion, a pseudo-condition called DEFINE, and assertions. - - Checking for a used subpattern by number - - If the text between the parentheses consists of a sequence of digits, - the condition is true if a capturing subpattern of that number has pre- - viously matched. If there is more than one capturing subpattern with - the same number (see the earlier section about duplicate subpattern - numbers), the condition is true if any of them have matched. An alter- - native notation is to precede the digits with a plus or minus sign. In - this case, the subpattern number is relative rather than absolute. The - most recently opened parentheses can be referenced by (?(-1), the next - most recent by (?(-2), and so on. Inside loops it can also make sense - to refer to subsequent groups. The next parentheses to be opened can be - referenced as (?(+1), and so on. (The value zero in any of these forms - is not used; it provokes a compile-time error.) - - Consider the following pattern, which contains non-significant white - space to make it more readable (assume the PCRE_EXTENDED option) and to - divide it into three parts for ease of discussion: - - ( \( )? [^()]+ (?(1) \) ) - - The first part matches an optional opening parenthesis, and if that - character is present, sets it as the first captured substring. The sec- - ond part matches one or more characters that are not parentheses. The - third part is a conditional subpattern that tests whether or not the - first set of parentheses matched. If they did, that is, if subject - started with an opening parenthesis, the condition is true, and so the - yes-pattern is executed and a closing parenthesis is required. Other- - wise, since no-pattern is not present, the subpattern matches nothing. - In other words, this pattern matches a sequence of non-parentheses, - optionally enclosed in parentheses. - - If you were embedding this pattern in a larger one, you could use a - relative reference: - - ...other stuff... ( \( )? [^()]+ (?(-1) \) ) ... - - This makes the fragment independent of the parentheses in the larger - pattern. - - Checking for a used subpattern by name - - Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a - used subpattern by name. For compatibility with earlier versions of - PCRE, which had this facility before Perl, the syntax (?(name)...) is - also recognized. - - Rewriting the above example to use a named subpattern gives this: - - (?<OPEN> \( )? [^()]+ (?(<OPEN>) \) ) - - If the name used in a condition of this kind is a duplicate, the test - is applied to all subpatterns of the same name, and is true if any one - of them has matched. - - Checking for pattern recursion - - If the condition is the string (R), and there is no subpattern with the - name R, the condition is true if a recursive call to the whole pattern - or any subpattern has been made. If digits or a name preceded by amper- - sand follow the letter R, for example: - - (?(R3)...) or (?(R&name)...) - - the condition is true if the most recent recursion is into a subpattern - whose number or name is given. This condition does not check the entire - recursion stack. If the name used in a condition of this kind is a - duplicate, the test is applied to all subpatterns of the same name, and - is true if any one of them is the most recent recursion. - - At "top level", all these recursion test conditions are false. The - syntax for recursive patterns is described below. - - Defining subpatterns for use by reference only - - If the condition is the string (DEFINE), and there is no subpattern - with the name DEFINE, the condition is always false. In this case, - there may be only one alternative in the subpattern. It is always - skipped if control reaches this point in the pattern; the idea of - DEFINE is that it can be used to define subroutines that can be refer- - enced from elsewhere. (The use of subroutines is described below.) For - example, a pattern to match an IPv4 address such as "192.168.23.245" - could be written like this (ignore white space and line breaks): - - (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) ) - \b (?&byte) (\.(?&byte)){3} \b - - The first part of the pattern is a DEFINE group inside which a another - group named "byte" is defined. This matches an individual component of - an IPv4 address (a number less than 256). When matching takes place, - this part of the pattern is skipped because DEFINE acts like a false - condition. The rest of the pattern uses references to the named group - to match the four dot-separated components of an IPv4 address, insist- - ing on a word boundary at each end. - - Assertion conditions - - If the condition is not in any of the above formats, it must be an - assertion. This may be a positive or negative lookahead or lookbehind - assertion. Consider this pattern, again containing non-significant - white space, and with the two alternatives on the second line: - - (?(?=[^a-z]*[a-z]) - \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) - - The condition is a positive lookahead assertion that matches an - optional sequence of non-letters followed by a letter. In other words, - it tests for the presence of at least one letter in the subject. If a - letter is found, the subject is matched against the first alternative; - otherwise it is matched against the second. This pattern matches - strings in one of the two forms dd-aaa-dd or dd-dd-dd, where aaa are - letters and dd are digits. - - -COMMENTS - - There are two ways of including comments in patterns that are processed - by PCRE. In both cases, the start of the comment must not be in a char- - acter class, nor in the middle of any other sequence of related charac- - ters such as (?: or a subpattern name or number. The characters that - make up a comment play no part in the pattern matching. - - The sequence (?# marks the start of a comment that continues up to the - next closing parenthesis. Nested parentheses are not permitted. If the - PCRE_EXTENDED option is set, an unescaped # character also introduces a - comment, which in this case continues to immediately after the next - newline character or character sequence in the pattern. Which charac- - ters are interpreted as newlines is controlled by the options passed to - a compiling function or by a special sequence at the start of the pat- - tern, as described in the section entitled "Newline conventions" above. - Note that the end of this type of comment is a literal newline sequence - in the pattern; escape sequences that happen to represent a newline do - not count. For example, consider this pattern when PCRE_EXTENDED is - set, and the default newline convention is in force: - - abc #comment \n still comment - - On encountering the # character, pcre_compile() skips along, looking - for a newline in the pattern. The sequence \n is still literal at this - stage, so it does not terminate the comment. Only an actual character - with the code value 0x0a (the default newline) does so. - - -RECURSIVE PATTERNS - - Consider the problem of matching a string in parentheses, allowing for - unlimited nested parentheses. Without the use of recursion, the best - that can be done is to use a pattern that matches up to some fixed - depth of nesting. It is not possible to handle an arbitrary nesting - depth. - - For some time, Perl has provided a facility that allows regular expres- - sions to recurse (amongst other things). It does this by interpolating - Perl code in the expression at run time, and the code can refer to the - expression itself. A Perl pattern using code interpolation to solve the - parentheses problem can be created like this: - - $re = qr{\( (?: (?>[^()]+) | (?p{$re}) )* \)}x; - - The (?p{...}) item interpolates Perl code at run time, and in this case - refers recursively to the pattern in which it appears. - - Obviously, PCRE cannot support the interpolation of Perl code. Instead, - it supports special syntax for recursion of the entire pattern, and - also for individual subpattern recursion. After its introduction in - PCRE and Python, this kind of recursion was subsequently introduced - into Perl at release 5.10. - - A special item that consists of (? followed by a number greater than - zero and a closing parenthesis is a recursive subroutine call of the - subpattern of the given number, provided that it occurs inside that - subpattern. (If not, it is a non-recursive subroutine call, which is - described in the next section.) The special item (?R) or (?0) is a - recursive call of the entire regular expression. - - This PCRE pattern solves the nested parentheses problem (assume the - PCRE_EXTENDED option is set so that white space is ignored): - - \( ( [^()]++ | (?R) )* \) - - First it matches an opening parenthesis. Then it matches any number of - substrings which can either be a sequence of non-parentheses, or a - recursive match of the pattern itself (that is, a correctly parenthe- - sized substring). Finally there is a closing parenthesis. Note the use - of a possessive quantifier to avoid backtracking into sequences of non- - parentheses. - - If this were part of a larger pattern, you would not want to recurse - the entire pattern, so instead you could use this: - - ( \( ( [^()]++ | (?1) )* \) ) - - We have put the pattern into parentheses, and caused the recursion to - refer to them instead of the whole pattern. - - In a larger pattern, keeping track of parenthesis numbers can be - tricky. This is made easier by the use of relative references. Instead - of (?1) in the pattern above you can write (?-2) to refer to the second - most recently opened parentheses preceding the recursion. In other - words, a negative number counts capturing parentheses leftwards from - the point at which it is encountered. - - It is also possible to refer to subsequently opened parentheses, by - writing references such as (?+2). However, these cannot be recursive - because the reference is not inside the parentheses that are refer- - enced. They are always non-recursive subroutine calls, as described in - the next section. - - An alternative approach is to use named parentheses instead. The Perl - syntax for this is (?&name); PCRE's earlier syntax (?P>name) is also - supported. We could rewrite the above example as follows: - - (?<pn> \( ( [^()]++ | (?&pn) )* \) ) - - If there is more than one subpattern with the same name, the earliest - one is used. - - This particular example pattern that we have been looking at contains - nested unlimited repeats, and so the use of a possessive quantifier for - matching strings of non-parentheses is important when applying the pat- - tern to strings that do not match. For example, when this pattern is - applied to - - (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() - - it yields "no match" quickly. However, if a possessive quantifier is - not used, the match runs for a very long time indeed because there are - so many different ways the + and * repeats can carve up the subject, - and all have to be tested before failure can be reported. - - At the end of a match, the values of capturing parentheses are those - from the outermost level. If you want to obtain intermediate values, a - callout function can be used (see below and the pcrecallout documenta- - tion). If the pattern above is matched against - - (ab(cd)ef) - - the value for the inner capturing parentheses (numbered 2) is "ef", - which is the last value taken on at the top level. If a capturing sub- - pattern is not matched at the top level, its final captured value is - unset, even if it was (temporarily) set at a deeper level during the - matching process. - - If there are more than 15 capturing parentheses in a pattern, PCRE has - to obtain extra memory to store data during a recursion, which it does - by using pcre_malloc, freeing it via pcre_free afterwards. If no memory - can be obtained, the match fails with the PCRE_ERROR_NOMEMORY error. - - Do not confuse the (?R) item with the condition (R), which tests for - recursion. Consider this pattern, which matches text in angle brack- - ets, allowing for arbitrary nesting. Only digits are allowed in nested - brackets (that is, when recursing), whereas any characters are permit- - ted at the outer level. - - < (?: (?(R) \d++ | [^<>]*+) | (?R)) * > - - In this pattern, (?(R) is the start of a conditional subpattern, with - two different alternatives for the recursive and non-recursive cases. - The (?R) item is the actual recursive call. - - Differences in recursion processing between PCRE and Perl - - Recursion processing in PCRE differs from Perl in two important ways. - In PCRE (like Python, but unlike Perl), a recursive subpattern call is - always treated as an atomic group. That is, once it has matched some of - the subject string, it is never re-entered, even if it contains untried - alternatives and there is a subsequent matching failure. This can be - illustrated by the following pattern, which purports to match a palin- - dromic string that contains an odd number of characters (for example, - "a", "aba", "abcba", "abcdcba"): - - ^(.|(.)(?1)\2)$ - - The idea is that it either matches a single character, or two identical - characters surrounding a sub-palindrome. In Perl, this pattern works; - in PCRE it does not if the pattern is longer than three characters. - Consider the subject string "abcba": - - At the top level, the first character is matched, but as it is not at - the end of the string, the first alternative fails; the second alterna- - tive is taken and the recursion kicks in. The recursive call to subpat- - tern 1 successfully matches the next character ("b"). (Note that the - beginning and end of line tests are not part of the recursion). - - Back at the top level, the next character ("c") is compared with what - subpattern 2 matched, which was "a". This fails. Because the recursion - is treated as an atomic group, there are now no backtracking points, - and so the entire match fails. (Perl is able, at this point, to re- - enter the recursion and try the second alternative.) However, if the - pattern is written with the alternatives in the other order, things are - different: - - ^((.)(?1)\2|.)$ - - This time, the recursing alternative is tried first, and continues to - recurse until it runs out of characters, at which point the recursion - fails. But this time we do have another alternative to try at the - higher level. That is the big difference: in the previous case the - remaining alternative is at a deeper recursion level, which PCRE cannot - use. - - To change the pattern so that it matches all palindromic strings, not - just those with an odd number of characters, it is tempting to change - the pattern to this: - - ^((.)(?1)\2|.?)$ - - Again, this works in Perl, but not in PCRE, and for the same reason. - When a deeper recursion has matched a single character, it cannot be - entered again in order to match an empty string. The solution is to - separate the two cases, and write out the odd and even cases as alter- - natives at the higher level: - - ^(?:((.)(?1)\2|)|((.)(?3)\4|.)) - - If you want to match typical palindromic phrases, the pattern has to - ignore all non-word characters, which can be done like this: - - ^\W*+(?:((.)\W*+(?1)\W*+\2|)|((.)\W*+(?3)\W*+\4|\W*+.\W*+))\W*+$ - - If run with the PCRE_CASELESS option, this pattern matches phrases such - as "A man, a plan, a canal: Panama!" and it works well in both PCRE and - Perl. Note the use of the possessive quantifier *+ to avoid backtrack- - ing into sequences of non-word characters. Without this, PCRE takes a - great deal longer (ten times or more) to match typical phrases, and - Perl takes so long that you think it has gone into a loop. - - WARNING: The palindrome-matching patterns above work only if the sub- - ject string does not start with a palindrome that is shorter than the - entire string. For example, although "abcba" is correctly matched, if - the subject is "ababa", PCRE finds the palindrome "aba" at the start, - then fails at top level because the end of the string does not follow. - Once again, it cannot jump back into the recursion to try other alter- - natives, so the entire match fails. - - The second way in which PCRE and Perl differ in their recursion pro- - cessing is in the handling of captured values. In Perl, when a subpat- - tern is called recursively or as a subpattern (see the next section), - it has no access to any values that were captured outside the recur- - sion, whereas in PCRE these values can be referenced. Consider this - pattern: - - ^(.)(\1|a(?2)) - - In PCRE, this pattern matches "bab". The first capturing parentheses - match "b", then in the second group, when the back reference \1 fails - to match "b", the second alternative matches "a" and then recurses. In - the recursion, \1 does now match "b" and so the whole match succeeds. - In Perl, the pattern fails to match because inside the recursive call - \1 cannot access the externally set value. - - -SUBPATTERNS AS SUBROUTINES - - If the syntax for a recursive subpattern call (either by number or by - name) is used outside the parentheses to which it refers, it operates - like a subroutine in a programming language. The called subpattern may - be defined before or after the reference. A numbered reference can be - absolute or relative, as in these examples: - - (...(absolute)...)...(?2)... - (...(relative)...)...(?-1)... - (...(?+1)...(relative)... - - An earlier example pointed out that the pattern - - (sens|respons)e and \1ibility - - matches "sense and sensibility" and "response and responsibility", but - not "sense and responsibility". If instead the pattern - - (sens|respons)e and (?1)ibility - - is used, it does match "sense and responsibility" as well as the other - two strings. Another example is given in the discussion of DEFINE - above. - - All subroutine calls, whether recursive or not, are always treated as - atomic groups. That is, once a subroutine has matched some of the sub- - ject string, it is never re-entered, even if it contains untried alter- - natives and there is a subsequent matching failure. Any capturing - parentheses that are set during the subroutine call revert to their - previous values afterwards. - - Processing options such as case-independence are fixed when a subpat- - tern is defined, so if it is used as a subroutine, such options cannot - be changed for different calls. For example, consider this pattern: - - (abc)(?i:(?-1)) - - It matches "abcabc". It does not match "abcABC" because the change of - processing option does not affect the called subpattern. - - -ONIGURUMA SUBROUTINE SYNTAX - - For compatibility with Oniguruma, the non-Perl syntax \g followed by a - name or a number enclosed either in angle brackets or single quotes, is - an alternative syntax for referencing a subpattern as a subroutine, - possibly recursively. Here are two of the examples used above, rewrit- - ten using this syntax: - - (?<pn> \( ( (?>[^()]+) | \g<pn> )* \) ) - (sens|respons)e and \g'1'ibility - - PCRE supports an extension to Oniguruma: if a number is preceded by a - plus or a minus sign it is taken as a relative reference. For example: - - (abc)(?i:\g<-1>) - - Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are not - synonymous. The former is a back reference; the latter is a subroutine - call. - - -CALLOUTS - - Perl has a feature whereby using the sequence (?{...}) causes arbitrary - Perl code to be obeyed in the middle of matching a regular expression. - This makes it possible, amongst other things, to extract different sub- - strings that match the same pair of parentheses when there is a repeti- - tion. - - PCRE provides a similar feature, but of course it cannot obey arbitrary - Perl code. The feature is called "callout". The caller of PCRE provides - an external function by putting its entry point in the global variable - pcre_callout (8-bit library) or pcre[16|32]_callout (16-bit or 32-bit - library). By default, this variable contains NULL, which disables all - calling out. - - Within a regular expression, (?C) indicates the points at which the - external function is to be called. If you want to identify different - callout points, you can put a number less than 256 after the letter C. - The default value is zero. For example, this pattern has two callout - points: - - (?C1)abc(?C2)def - - If the PCRE_AUTO_CALLOUT flag is passed to a compiling function, call- - outs are automatically installed before each item in the pattern. They - are all numbered 255. If there is a conditional group in the pattern - whose condition is an assertion, an additional callout is inserted just - before the condition. An explicit callout may also be set at this posi- - tion, as in this example: - - (?(?C9)(?=a)abc|def) - - Note that this applies only to assertion conditions, not to other types - of condition. - - During matching, when PCRE reaches a callout point, the external func- - tion is called. It is provided with the number of the callout, the - position in the pattern, and, optionally, one item of data originally - supplied by the caller of the matching function. The callout function - may cause matching to proceed, to backtrack, or to fail altogether. - - By default, PCRE implements a number of optimizations at compile time - and matching time, and one side-effect is that sometimes callouts are - skipped. If you need all possible callouts to happen, you need to set - options that disable the relevant optimizations. More details, and a - complete description of the interface to the callout function, are - given in the pcrecallout documentation. - - -BACKTRACKING CONTROL - - Perl 5.10 introduced a number of "Special Backtracking Control Verbs", - which are still described in the Perl documentation as "experimental - and subject to change or removal in a future version of Perl". It goes - on to say: "Their usage in production code should be noted to avoid - problems during upgrades." The same remarks apply to the PCRE features - described in this section. - - The new verbs make use of what was previously invalid syntax: an open- - ing parenthesis followed by an asterisk. They are generally of the form - (*VERB) or (*VERB:NAME). Some may take either form, possibly behaving - differently depending on whether or not a name is present. A name is - any sequence of characters that does not include a closing parenthesis. - The maximum length of name is 255 in the 8-bit library and 65535 in the - 16-bit and 32-bit libraries. If the name is empty, that is, if the - closing parenthesis immediately follows the colon, the effect is as if - the colon were not there. Any number of these verbs may occur in a - pattern. - - Since these verbs are specifically related to backtracking, most of - them can be used only when the pattern is to be matched using one of - the traditional matching functions, because these use a backtracking - algorithm. With the exception of (*FAIL), which behaves like a failing - negative assertion, the backtracking control verbs cause an error if - encountered by a DFA matching function. - - The behaviour of these verbs in repeated groups, assertions, and in - subpatterns called as subroutines (whether or not recursively) is docu- - mented below. - - Optimizations that affect backtracking verbs - - PCRE contains some optimizations that are used to speed up matching by - running some checks at the start of each match attempt. For example, it - may know the minimum length of matching subject, or that a particular - character must be present. When one of these optimizations bypasses the - running of a match, any included backtracking verbs will not, of - course, be processed. You can suppress the start-of-match optimizations - by setting the PCRE_NO_START_OPTIMIZE option when calling pcre_com- - pile() or pcre_exec(), or by starting the pattern with (*NO_START_OPT). - There is more discussion of this option in the section entitled "Option - bits for pcre_exec()" in the pcreapi documentation. - - Experiments with Perl suggest that it too has similar optimizations, - sometimes leading to anomalous results. - - Verbs that act immediately - - The following verbs act as soon as they are encountered. They may not - be followed by a name. - - (*ACCEPT) - - This verb causes the match to end successfully, skipping the remainder - of the pattern. However, when it is inside a subpattern that is called - as a subroutine, only that subpattern is ended successfully. Matching - then continues at the outer level. If (*ACCEPT) in triggered in a posi- - tive assertion, the assertion succeeds; in a negative assertion, the - assertion fails. - - If (*ACCEPT) is inside capturing parentheses, the data so far is cap- - tured. For example: - - A((?:A|B(*ACCEPT)|C)D) - - This matches "AB", "AAD", or "ACD"; when it matches "AB", "B" is cap- - tured by the outer parentheses. - - (*FAIL) or (*F) - - This verb causes a matching failure, forcing backtracking to occur. It - is equivalent to (?!) but easier to read. The Perl documentation notes - that it is probably useful only when combined with (?{}) or (??{}). - Those are, of course, Perl features that are not present in PCRE. The - nearest equivalent is the callout feature, as for example in this pat- - tern: - - a+(?C)(*FAIL) - - A match with the string "aaaa" always fails, but the callout is taken - before each backtrack happens (in this example, 10 times). - - Recording which path was taken - - There is one verb whose main purpose is to track how a match was - arrived at, though it also has a secondary use in conjunction with - advancing the match starting point (see (*SKIP) below). - - (*MARK:NAME) or (*:NAME) - - A name is always required with this verb. There may be as many - instances of (*MARK) as you like in a pattern, and their names do not - have to be unique. - - When a match succeeds, the name of the last-encountered (*MARK:NAME), - (*PRUNE:NAME), or (*THEN:NAME) on the matching path is passed back to - the caller as described in the section entitled "Extra data for - pcre_exec()" in the pcreapi documentation. Here is an example of - pcretest output, where the /K modifier requests the retrieval and out- - putting of (*MARK) data: - - re> /X(*MARK:A)Y|X(*MARK:B)Z/K - data> XY - 0: XY - MK: A - XZ - 0: XZ - MK: B - - The (*MARK) name is tagged with "MK:" in this output, and in this exam- - ple it indicates which of the two alternatives matched. This is a more - efficient way of obtaining this information than putting each alterna- - tive in its own capturing parentheses. - - If a verb with a name is encountered in a positive assertion that is - true, the name is recorded and passed back if it is the last-encoun- - tered. This does not happen for negative assertions or failing positive - assertions. - - After a partial match or a failed match, the last encountered name in - the entire match process is returned. For example: - - re> /X(*MARK:A)Y|X(*MARK:B)Z/K - data> XP - No match, mark = B - - Note that in this unanchored example the mark is retained from the - match attempt that started at the letter "X" in the subject. Subsequent - match attempts starting at "P" and then with an empty string do not get - as far as the (*MARK) item, but nevertheless do not reset it. - - If you are interested in (*MARK) values after failed matches, you - should probably set the PCRE_NO_START_OPTIMIZE option (see above) to - ensure that the match is always attempted. - - Verbs that act after backtracking - - The following verbs do nothing when they are encountered. Matching con- - tinues with what follows, but if there is no subsequent match, causing - a backtrack to the verb, a failure is forced. That is, backtracking - cannot pass to the left of the verb. However, when one of these verbs - appears inside an atomic group or an assertion that is true, its effect - is confined to that group, because once the group has been matched, - there is never any backtracking into it. In this situation, backtrack- - ing can "jump back" to the left of the entire atomic group or asser- - tion. (Remember also, as stated above, that this localization also - applies in subroutine calls.) - - These verbs differ in exactly what kind of failure occurs when back- - tracking reaches them. The behaviour described below is what happens - when the verb is not in a subroutine or an assertion. Subsequent sec- - tions cover these special cases. - - (*COMMIT) - - This verb, which may not be followed by a name, causes the whole match - to fail outright if there is a later matching failure that causes back- - tracking to reach it. Even if the pattern is unanchored, no further - attempts to find a match by advancing the starting point take place. If - (*COMMIT) is the only backtracking verb that is encountered, once it - has been passed pcre_exec() is committed to finding a match at the cur- - rent starting point, or not at all. For example: - - a+(*COMMIT)b - - This matches "xxaab" but not "aacaab". It can be thought of as a kind - of dynamic anchor, or "I've started, so I must finish." The name of the - most recently passed (*MARK) in the path is passed back when (*COMMIT) - forces a match failure. - - If there is more than one backtracking verb in a pattern, a different - one that follows (*COMMIT) may be triggered first, so merely passing - (*COMMIT) during a match does not always guarantee that a match must be - at this starting point. - - Note that (*COMMIT) at the start of a pattern is not the same as an - anchor, unless PCRE's start-of-match optimizations are turned off, as - shown in this output from pcretest: - - re> /(*COMMIT)abc/ - data> xyzabc - 0: abc - data> xyzabc\Y - No match - - For this pattern, PCRE knows that any match must start with "a", so the - optimization skips along the subject to "a" before applying the pattern - to the first set of data. The match attempt then succeeds. In the sec- - ond set of data, the escape sequence \Y is interpreted by the pcretest - program. It causes the PCRE_NO_START_OPTIMIZE option to be set when - pcre_exec() is called. This disables the optimization that skips along - to the first character. The pattern is now applied starting at "x", and - so the (*COMMIT) causes the match to fail without trying any other - starting points. - - (*PRUNE) or (*PRUNE:NAME) - - This verb causes the match to fail at the current starting position in - the subject if there is a later matching failure that causes backtrack- - ing to reach it. If the pattern is unanchored, the normal "bumpalong" - advance to the next starting character then happens. Backtracking can - occur as usual to the left of (*PRUNE), before it is reached, or when - matching to the right of (*PRUNE), but if there is no match to the - right, backtracking cannot cross (*PRUNE). In simple cases, the use of - (*PRUNE) is just an alternative to an atomic group or possessive quan- - tifier, but there are some uses of (*PRUNE) that cannot be expressed in - any other way. In an anchored pattern (*PRUNE) has the same effect as - (*COMMIT). - - The behaviour of (*PRUNE:NAME) is the not the same as - (*MARK:NAME)(*PRUNE). It is like (*MARK:NAME) in that the name is - remembered for passing back to the caller. However, (*SKIP:NAME) - searches only for names set with (*MARK). - - (*SKIP) - - This verb, when given without a name, is like (*PRUNE), except that if - the pattern is unanchored, the "bumpalong" advance is not to the next - character, but to the position in the subject where (*SKIP) was encoun- - tered. (*SKIP) signifies that whatever text was matched leading up to - it cannot be part of a successful match. Consider: - - a+(*SKIP)b - - If the subject is "aaaac...", after the first match attempt fails - (starting at the first character in the string), the starting point - skips on to start the next attempt at "c". Note that a possessive quan- - tifer does not have the same effect as this example; although it would - suppress backtracking during the first match attempt, the second - attempt would start at the second character instead of skipping on to - "c". - - (*SKIP:NAME) - - When (*SKIP) has an associated name, its behaviour is modified. When it - is triggered, the previous path through the pattern is searched for the - most recent (*MARK) that has the same name. If one is found, the - "bumpalong" advance is to the subject position that corresponds to that - (*MARK) instead of to where (*SKIP) was encountered. If no (*MARK) with - a matching name is found, the (*SKIP) is ignored. - - Note that (*SKIP:NAME) searches only for names set by (*MARK:NAME). It - ignores names that are set by (*PRUNE:NAME) or (*THEN:NAME). - - (*THEN) or (*THEN:NAME) - - This verb causes a skip to the next innermost alternative when back- - tracking reaches it. That is, it cancels any further backtracking - within the current alternative. Its name comes from the observation - that it can be used for a pattern-based if-then-else block: - - ( COND1 (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ ) ... - - If the COND1 pattern matches, FOO is tried (and possibly further items - after the end of the group if FOO succeeds); on failure, the matcher - skips to the second alternative and tries COND2, without backtracking - into COND1. If that succeeds and BAR fails, COND3 is tried. If subse- - quently BAZ fails, there are no more alternatives, so there is a back- - track to whatever came before the entire group. If (*THEN) is not - inside an alternation, it acts like (*PRUNE). - - The behaviour of (*THEN:NAME) is the not the same as - (*MARK:NAME)(*THEN). It is like (*MARK:NAME) in that the name is - remembered for passing back to the caller. However, (*SKIP:NAME) - searches only for names set with (*MARK). - - A subpattern that does not contain a | character is just a part of the - enclosing alternative; it is not a nested alternation with only one - alternative. The effect of (*THEN) extends beyond such a subpattern to - the enclosing alternative. Consider this pattern, where A, B, etc. are - complex pattern fragments that do not contain any | characters at this - level: - - A (B(*THEN)C) | D - - If A and B are matched, but there is a failure in C, matching does not - backtrack into A; instead it moves to the next alternative, that is, D. - However, if the subpattern containing (*THEN) is given an alternative, - it behaves differently: - - A (B(*THEN)C | (*FAIL)) | D - - The effect of (*THEN) is now confined to the inner subpattern. After a - failure in C, matching moves to (*FAIL), which causes the whole subpat- - tern to fail because there are no more alternatives to try. In this - case, matching does now backtrack into A. - - Note that a conditional subpattern is not considered as having two - alternatives, because only one is ever used. In other words, the | - character in a conditional subpattern has a different meaning. Ignoring - white space, consider: - - ^.*? (?(?=a) a | b(*THEN)c ) - - If the subject is "ba", this pattern does not match. Because .*? is - ungreedy, it initially matches zero characters. The condition (?=a) - then fails, the character "b" is matched, but "c" is not. At this - point, matching does not backtrack to .*? as might perhaps be expected - from the presence of the | character. The conditional subpattern is - part of the single alternative that comprises the whole pattern, and so - the match fails. (If there was a backtrack into .*?, allowing it to - match "b", the match would succeed.) - - The verbs just described provide four different "strengths" of control - when subsequent matching fails. (*THEN) is the weakest, carrying on the - match at the next alternative. (*PRUNE) comes next, failing the match - at the current starting position, but allowing an advance to the next - character (for an unanchored pattern). (*SKIP) is similar, except that - the advance may be more than one character. (*COMMIT) is the strongest, - causing the entire match to fail. - - More than one backtracking verb - - If more than one backtracking verb is present in a pattern, the one - that is backtracked onto first acts. For example, consider this pat- - tern, where A, B, etc. are complex pattern fragments: - - (A(*COMMIT)B(*THEN)C|ABD) - - If A matches but B fails, the backtrack to (*COMMIT) causes the entire - match to fail. However, if A and B match, but C fails, the backtrack to - (*THEN) causes the next alternative (ABD) to be tried. This behaviour - is consistent, but is not always the same as Perl's. It means that if - two or more backtracking verbs appear in succession, all the the last - of them has no effect. Consider this example: - - ...(*COMMIT)(*PRUNE)... - - If there is a matching failure to the right, backtracking onto (*PRUNE) - causes it to be triggered, and its action is taken. There can never be - a backtrack onto (*COMMIT). - - Backtracking verbs in repeated groups - - PCRE differs from Perl in its handling of backtracking verbs in - repeated groups. For example, consider: - - /(a(*COMMIT)b)+ac/ - - If the subject is "abac", Perl matches, but PCRE fails because the - (*COMMIT) in the second repeat of the group acts. - - Backtracking verbs in assertions - - (*FAIL) in an assertion has its normal effect: it forces an immediate - backtrack. - - (*ACCEPT) in a positive assertion causes the assertion to succeed with- - out any further processing. In a negative assertion, (*ACCEPT) causes - the assertion to fail without any further processing. - - The other backtracking verbs are not treated specially if they appear - in a positive assertion. In particular, (*THEN) skips to the next - alternative in the innermost enclosing group that has alternations, - whether or not this is within the assertion. - - Negative assertions are, however, different, in order to ensure that - changing a positive assertion into a negative assertion changes its - result. Backtracking into (*COMMIT), (*SKIP), or (*PRUNE) causes a neg- - ative assertion to be true, without considering any further alternative - branches in the assertion. Backtracking into (*THEN) causes it to skip - to the next enclosing alternative within the assertion (the normal be- - haviour), but if the assertion does not have such an alternative, - (*THEN) behaves like (*PRUNE). - - Backtracking verbs in subroutines - - These behaviours occur whether or not the subpattern is called recur- - sively. Perl's treatment of subroutines is different in some cases. - - (*FAIL) in a subpattern called as a subroutine has its normal effect: - it forces an immediate backtrack. - - (*ACCEPT) in a subpattern called as a subroutine causes the subroutine - match to succeed without any further processing. Matching then contin- - ues after the subroutine call. - - (*COMMIT), (*SKIP), and (*PRUNE) in a subpattern called as a subroutine - cause the subroutine match to fail. - - (*THEN) skips to the next alternative in the innermost enclosing group - within the subpattern that has alternatives. If there is no such group - within the subpattern, (*THEN) causes the subroutine match to fail. - - -SEE ALSO - - pcreapi(3), pcrecallout(3), pcrematching(3), pcresyntax(3), pcre(3), - pcre16(3), pcre32(3). - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 08 January 2014 - Copyright (c) 1997-2014 University of Cambridge. ------------------------------------------------------------------------------- - - -PCRESYNTAX(3) Library Functions Manual PCRESYNTAX(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -PCRE REGULAR EXPRESSION SYNTAX SUMMARY - - The full syntax and semantics of the regular expressions that are sup- - ported by PCRE are described in the pcrepattern documentation. This - document contains a quick-reference summary of the syntax. - - -QUOTING - - \x where x is non-alphanumeric is a literal x - \Q...\E treat enclosed characters as literal - - -CHARACTERS - - \a alarm, that is, the BEL character (hex 07) - \cx "control-x", where x is any ASCII character - \e escape (hex 1B) - \f form feed (hex 0C) - \n newline (hex 0A) - \r carriage return (hex 0D) - \t tab (hex 09) - \0dd character with octal code 0dd - \ddd character with octal code ddd, or backreference - \o{ddd..} character with octal code ddd.. - \xhh character with hex code hh - \x{hhh..} character with hex code hhh.. - - Note that \0dd is always an octal code, and that \8 and \9 are the lit- - eral characters "8" and "9". - - -CHARACTER TYPES - - . any character except newline; - in dotall mode, any character whatsoever - \C one data unit, even in UTF mode (best avoided) - \d a decimal digit - \D a character that is not a decimal digit - \h a horizontal white space character - \H a character that is not a horizontal white space character - \N a character that is not a newline - \p{xx} a character with the xx property - \P{xx} a character without the xx property - \R a newline sequence - \s a white space character - \S a character that is not a white space character - \v a vertical white space character - \V a character that is not a vertical white space character - \w a "word" character - \W a "non-word" character - \X a Unicode extended grapheme cluster - - By default, \d, \s, and \w match only ASCII characters, even in UTF-8 - mode or in the 16- bit and 32-bit libraries. However, if locale-spe- - cific matching is happening, \s and \w may also match characters with - code points in the range 128-255. If the PCRE_UCP option is set, the - behaviour of these escape sequences is changed to use Unicode proper- - ties and they match many more characters. - - -GENERAL CATEGORY PROPERTIES FOR \p and \P - - C Other - Cc Control - Cf Format - Cn Unassigned - Co Private use - Cs Surrogate - - L Letter - Ll Lower case letter - Lm Modifier letter - Lo Other letter - Lt Title case letter - Lu Upper case letter - L& Ll, Lu, or Lt - - M Mark - Mc Spacing mark - Me Enclosing mark - Mn Non-spacing mark - - N Number - Nd Decimal number - Nl Letter number - No Other number - - P Punctuation - Pc Connector punctuation - Pd Dash punctuation - Pe Close punctuation - Pf Final punctuation - Pi Initial punctuation - Po Other punctuation - Ps Open punctuation - - S Symbol - Sc Currency symbol - Sk Modifier symbol - Sm Mathematical symbol - So Other symbol - - Z Separator - Zl Line separator - Zp Paragraph separator - Zs Space separator - - -PCRE SPECIAL CATEGORY PROPERTIES FOR \p and \P - - Xan Alphanumeric: union of properties L and N - Xps POSIX space: property Z or tab, NL, VT, FF, CR - Xsp Perl space: property Z or tab, NL, VT, FF, CR - Xuc Univerally-named character: one that can be - represented by a Universal Character Name - Xwd Perl word: property Xan or underscore - - Perl and POSIX space are now the same. Perl added VT to its space char- - acter set at release 5.18 and PCRE changed at release 8.34. - - -SCRIPT NAMES FOR \p AND \P - - Arabic, Armenian, Avestan, Balinese, Bamum, Batak, Bengali, Bopomofo, - Brahmi, Braille, Buginese, Buhid, Canadian_Aboriginal, Carian, Chakma, - Cham, Cherokee, Common, Coptic, Cuneiform, Cypriot, Cyrillic, Deseret, - Devanagari, Egyptian_Hieroglyphs, Ethiopic, Georgian, Glagolitic, - Gothic, Greek, Gujarati, Gurmukhi, Han, Hangul, Hanunoo, Hebrew, Hira- - gana, Imperial_Aramaic, Inherited, Inscriptional_Pahlavi, Inscrip- - tional_Parthian, Javanese, Kaithi, Kannada, Katakana, Kayah_Li, - Kharoshthi, Khmer, Lao, Latin, Lepcha, Limbu, Linear_B, Lisu, Lycian, - Lydian, Malayalam, Mandaic, Meetei_Mayek, Meroitic_Cursive, - Meroitic_Hieroglyphs, Miao, Mongolian, Myanmar, New_Tai_Lue, Nko, - Ogham, Old_Italic, Old_Persian, Old_South_Arabian, Old_Turkic, - Ol_Chiki, Oriya, Osmanya, Phags_Pa, Phoenician, Rejang, Runic, Samari- - tan, Saurashtra, Sharada, Shavian, Sinhala, Sora_Sompeng, Sundanese, - Syloti_Nagri, Syriac, Tagalog, Tagbanwa, Tai_Le, Tai_Tham, Tai_Viet, - Takri, Tamil, Telugu, Thaana, Thai, Tibetan, Tifinagh, Ugaritic, Vai, - Yi. - - -CHARACTER CLASSES - - [...] positive character class - [^...] negative character class - [x-y] range (can be used for hex characters) - [[:xxx:]] positive POSIX named set - [[:^xxx:]] negative POSIX named set - - alnum alphanumeric - alpha alphabetic - ascii 0-127 - blank space or tab - cntrl control character - digit decimal digit - graph printing, excluding space - lower lower case letter - print printing, including space - punct printing, excluding alphanumeric - space white space - upper upper case letter - word same as \w - xdigit hexadecimal digit - - In PCRE, POSIX character set names recognize only ASCII characters by - default, but some of them use Unicode properties if PCRE_UCP is set. - You can use \Q...\E inside a character class. - - -QUANTIFIERS - - ? 0 or 1, greedy - ?+ 0 or 1, possessive - ?? 0 or 1, lazy - * 0 or more, greedy - *+ 0 or more, possessive - *? 0 or more, lazy - + 1 or more, greedy - ++ 1 or more, possessive - +? 1 or more, lazy - {n} exactly n - {n,m} at least n, no more than m, greedy - {n,m}+ at least n, no more than m, possessive - {n,m}? at least n, no more than m, lazy - {n,} n or more, greedy - {n,}+ n or more, possessive - {n,}? n or more, lazy - - -ANCHORS AND SIMPLE ASSERTIONS - - \b word boundary - \B not a word boundary - ^ start of subject - also after internal newline in multiline mode - \A start of subject - $ end of subject - also before newline at end of subject - also before internal newline in multiline mode - \Z end of subject - also before newline at end of subject - \z end of subject - \G first matching position in subject - - -MATCH POINT RESET - - \K reset start of match - - \K is honoured in positive assertions, but ignored in negative ones. - - -ALTERNATION - - expr|expr|expr... - - -CAPTURING - - (...) capturing group - (?<name>...) named capturing group (Perl) - (?'name'...) named capturing group (Perl) - (?P<name>...) named capturing group (Python) - (?:...) non-capturing group - (?|...) non-capturing group; reset group numbers for - capturing groups in each alternative - - -ATOMIC GROUPS - - (?>...) atomic, non-capturing group - - -COMMENT - - (?#....) comment (not nestable) - - -OPTION SETTING - - (?i) caseless - (?J) allow duplicate names - (?m) multiline - (?s) single line (dotall) - (?U) default ungreedy (lazy) - (?x) extended (ignore white space) - (?-...) unset option(s) - - The following are recognized only at the very start of a pattern or - after one of the newline or \R options with similar syntax. More than - one of them may appear. - - (*LIMIT_MATCH=d) set the match limit to d (decimal number) - (*LIMIT_RECURSION=d) set the recursion limit to d (decimal number) - (*NO_AUTO_POSSESS) no auto-possessification (PCRE_NO_AUTO_POSSESS) - (*NO_START_OPT) no start-match optimization (PCRE_NO_START_OPTIMIZE) - (*UTF8) set UTF-8 mode: 8-bit library (PCRE_UTF8) - (*UTF16) set UTF-16 mode: 16-bit library (PCRE_UTF16) - (*UTF32) set UTF-32 mode: 32-bit library (PCRE_UTF32) - (*UTF) set appropriate UTF mode for the library in use - (*UCP) set PCRE_UCP (use Unicode properties for \d etc) - - Note that LIMIT_MATCH and LIMIT_RECURSION can only reduce the value of - the limits set by the caller of pcre_exec(), not increase them. - - -NEWLINE CONVENTION - - These are recognized only at the very start of the pattern or after - option settings with a similar syntax. - - (*CR) carriage return only - (*LF) linefeed only - (*CRLF) carriage return followed by linefeed - (*ANYCRLF) all three of the above - (*ANY) any Unicode newline sequence - - -WHAT \R MATCHES - - These are recognized only at the very start of the pattern or after - option setting with a similar syntax. - - (*BSR_ANYCRLF) CR, LF, or CRLF - (*BSR_UNICODE) any Unicode newline sequence - - -LOOKAHEAD AND LOOKBEHIND ASSERTIONS - - (?=...) positive look ahead - (?!...) negative look ahead - (?<=...) positive look behind - (?<!...) negative look behind - - Each top-level branch of a look behind must be of a fixed length. - - -BACKREFERENCES - - \n reference by number (can be ambiguous) - \gn reference by number - \g{n} reference by number - \g{-n} relative reference by number - \k<name> reference by name (Perl) - \k'name' reference by name (Perl) - \g{name} reference by name (Perl) - \k{name} reference by name (.NET) - (?P=name) reference by name (Python) - - -SUBROUTINE REFERENCES (POSSIBLY RECURSIVE) - - (?R) recurse whole pattern - (?n) call subpattern by absolute number - (?+n) call subpattern by relative number - (?-n) call subpattern by relative number - (?&name) call subpattern by name (Perl) - (?P>name) call subpattern by name (Python) - \g<name> call subpattern by name (Oniguruma) - \g'name' call subpattern by name (Oniguruma) - \g<n> call subpattern by absolute number (Oniguruma) - \g'n' call subpattern by absolute number (Oniguruma) - \g<+n> call subpattern by relative number (PCRE extension) - \g'+n' call subpattern by relative number (PCRE extension) - \g<-n> call subpattern by relative number (PCRE extension) - \g'-n' call subpattern by relative number (PCRE extension) - - -CONDITIONAL PATTERNS - - (?(condition)yes-pattern) - (?(condition)yes-pattern|no-pattern) - - (?(n)... absolute reference condition - (?(+n)... relative reference condition - (?(-n)... relative reference condition - (?(<name>)... named reference condition (Perl) - (?('name')... named reference condition (Perl) - (?(name)... named reference condition (PCRE) - (?(R)... overall recursion condition - (?(Rn)... specific group recursion condition - (?(R&name)... specific recursion condition - (?(DEFINE)... define subpattern for reference - (?(assert)... assertion condition - - -BACKTRACKING CONTROL - - The following act immediately they are reached: - - (*ACCEPT) force successful match - (*FAIL) force backtrack; synonym (*F) - (*MARK:NAME) set name to be passed back; synonym (*:NAME) - - The following act only when a subsequent match failure causes a back- - track to reach them. They all force a match failure, but they differ in - what happens afterwards. Those that advance the start-of-match point do - so only if the pattern is not anchored. - - (*COMMIT) overall failure, no advance of starting point - (*PRUNE) advance to next starting character - (*PRUNE:NAME) equivalent to (*MARK:NAME)(*PRUNE) - (*SKIP) advance to current matching position - (*SKIP:NAME) advance to position corresponding to an earlier - (*MARK:NAME); if not found, the (*SKIP) is ignored - (*THEN) local failure, backtrack to next alternation - (*THEN:NAME) equivalent to (*MARK:NAME)(*THEN) - - -CALLOUTS - - (?C) callout - (?Cn) callout with data n - - -SEE ALSO - - pcrepattern(3), pcreapi(3), pcrecallout(3), pcrematching(3), pcre(3). - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 08 January 2014 - Copyright (c) 1997-2014 University of Cambridge. ------------------------------------------------------------------------------- - - -PCREUNICODE(3) Library Functions Manual PCREUNICODE(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -UTF-8, UTF-16, UTF-32, AND UNICODE PROPERTY SUPPORT - - As well as UTF-8 support, PCRE also supports UTF-16 (from release 8.30) - and UTF-32 (from release 8.32), by means of two additional libraries. - They can be built as well as, or instead of, the 8-bit library. - - -UTF-8 SUPPORT - - In order process UTF-8 strings, you must build PCRE's 8-bit library - with UTF support, and, in addition, you must call pcre_compile() with - the PCRE_UTF8 option flag, or the pattern must start with the sequence - (*UTF8) or (*UTF). When either of these is the case, both the pattern - and any subject strings that are matched against it are treated as - UTF-8 strings instead of strings of individual 1-byte characters. - - -UTF-16 AND UTF-32 SUPPORT - - In order process UTF-16 or UTF-32 strings, you must build PCRE's 16-bit - or 32-bit library with UTF support, and, in addition, you must call - pcre16_compile() or pcre32_compile() with the PCRE_UTF16 or PCRE_UTF32 - option flag, as appropriate. Alternatively, the pattern must start with - the sequence (*UTF16), (*UTF32), as appropriate, or (*UTF), which can - be used with either library. When UTF mode is set, both the pattern and - any subject strings that are matched against it are treated as UTF-16 - or UTF-32 strings instead of strings of individual 16-bit or 32-bit - characters. - - -UTF SUPPORT OVERHEAD - - If you compile PCRE with UTF support, but do not use it at run time, - the library will be a bit bigger, but the additional run time overhead - is limited to testing the PCRE_UTF[8|16|32] flag occasionally, so - should not be very big. - - -UNICODE PROPERTY SUPPORT - - If PCRE is built with Unicode character property support (which implies - UTF support), the escape sequences \p{..}, \P{..}, and \X can be used. - The available properties that can be tested are limited to the general - category properties such as Lu for an upper case letter or Nd for a - decimal number, the Unicode script names such as Arabic or Han, and the - derived properties Any and L&. Full lists is given in the pcrepattern - and pcresyntax documentation. Only the short names for properties are - supported. For example, \p{L} matches a letter. Its Perl synonym, - \p{Letter}, is not supported. Furthermore, in Perl, many properties - may optionally be prefixed by "Is", for compatibility with Perl 5.6. - PCRE does not support this. - - Validity of UTF-8 strings - - When you set the PCRE_UTF8 flag, the byte strings passed as patterns - and subjects are (by default) checked for validity on entry to the rel- - evant functions. The entire string is checked before any other process- - ing takes place. From release 7.3 of PCRE, the check is according the - rules of RFC 3629, which are themselves derived from the Unicode speci- - fication. Earlier releases of PCRE followed the rules of RFC 2279, - which allows the full range of 31-bit values (0 to 0x7FFFFFFF). The - current check allows only values in the range U+0 to U+10FFFF, exclud- - ing the surrogate area. (From release 8.33 the so-called "non-charac- - ter" code points are no longer excluded because Unicode corrigendum #9 - makes it clear that they should not be.) - - Characters in the "Surrogate Area" of Unicode are reserved for use by - UTF-16, where they are used in pairs to encode codepoints with values - greater than 0xFFFF. The code points that are encoded by UTF-16 pairs - are available independently in the UTF-8 and UTF-32 encodings. (In - other words, the whole surrogate thing is a fudge for UTF-16 which - unfortunately messes up UTF-8 and UTF-32.) - - If an invalid UTF-8 string is passed to PCRE, an error return is given. - At compile time, the only additional information is the offset to the - first byte of the failing character. The run-time functions pcre_exec() - and pcre_dfa_exec() also pass back this information, as well as a more - detailed reason code if the caller has provided memory in which to do - this. - - In some situations, you may already know that your strings are valid, - and therefore want to skip these checks in order to improve perfor- - mance, for example in the case of a long subject string that is being - scanned repeatedly. If you set the PCRE_NO_UTF8_CHECK flag at compile - time or at run time, PCRE assumes that the pattern or subject it is - given (respectively) contains only valid UTF-8 codes. In this case, it - does not diagnose an invalid UTF-8 string. - - Note that passing PCRE_NO_UTF8_CHECK to pcre_compile() just disables - the check for the pattern; it does not also apply to subject strings. - If you want to disable the check for a subject string you must pass - this option to pcre_exec() or pcre_dfa_exec(). - - If you pass an invalid UTF-8 string when PCRE_NO_UTF8_CHECK is set, the - result is undefined and your program may crash. - - Validity of UTF-16 strings - - When you set the PCRE_UTF16 flag, the strings of 16-bit data units that - are passed as patterns and subjects are (by default) checked for valid- - ity on entry to the relevant functions. Values other than those in the - surrogate range U+D800 to U+DFFF are independent code points. Values in - the surrogate range must be used in pairs in the correct manner. - - If an invalid UTF-16 string is passed to PCRE, an error return is - given. At compile time, the only additional information is the offset - to the first data unit of the failing character. The run-time functions - pcre16_exec() and pcre16_dfa_exec() also pass back this information, as - well as a more detailed reason code if the caller has provided memory - in which to do this. - - In some situations, you may already know that your strings are valid, - and therefore want to skip these checks in order to improve perfor- - mance. If you set the PCRE_NO_UTF16_CHECK flag at compile time or at - run time, PCRE assumes that the pattern or subject it is given (respec- - tively) contains only valid UTF-16 sequences. In this case, it does not - diagnose an invalid UTF-16 string. However, if an invalid string is - passed, the result is undefined. - - Validity of UTF-32 strings - - When you set the PCRE_UTF32 flag, the strings of 32-bit data units that - are passed as patterns and subjects are (by default) checked for valid- - ity on entry to the relevant functions. This check allows only values - in the range U+0 to U+10FFFF, excluding the surrogate area U+D800 to - U+DFFF. - - If an invalid UTF-32 string is passed to PCRE, an error return is - given. At compile time, the only additional information is the offset - to the first data unit of the failing character. The run-time functions - pcre32_exec() and pcre32_dfa_exec() also pass back this information, as - well as a more detailed reason code if the caller has provided memory - in which to do this. - - In some situations, you may already know that your strings are valid, - and therefore want to skip these checks in order to improve perfor- - mance. If you set the PCRE_NO_UTF32_CHECK flag at compile time or at - run time, PCRE assumes that the pattern or subject it is given (respec- - tively) contains only valid UTF-32 sequences. In this case, it does not - diagnose an invalid UTF-32 string. However, if an invalid string is - passed, the result is undefined. - - General comments about UTF modes - - 1. Codepoints less than 256 can be specified in patterns by either - braced or unbraced hexadecimal escape sequences (for example, \x{b3} or - \xb3). Larger values have to use braced sequences. - - 2. Octal numbers up to \777 are recognized, and in UTF-8 mode they - match two-byte characters for values greater than \177. - - 3. Repeat quantifiers apply to complete UTF characters, not to individ- - ual data units, for example: \x{100}{3}. - - 4. The dot metacharacter matches one UTF character instead of a single - data unit. - - 5. The escape sequence \C can be used to match a single byte in UTF-8 - mode, or a single 16-bit data unit in UTF-16 mode, or a single 32-bit - data unit in UTF-32 mode, but its use can lead to some strange effects - because it breaks up multi-unit characters (see the description of \C - in the pcrepattern documentation). The use of \C is not supported in - the alternative matching function pcre[16|32]_dfa_exec(), nor is it - supported in UTF mode by the JIT optimization of pcre[16|32]_exec(). If - JIT optimization is requested for a UTF pattern that contains \C, it - will not succeed, and so the matching will be carried out by the normal - interpretive function. - - 6. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly - test characters of any code value, but, by default, the characters that - PCRE recognizes as digits, spaces, or word characters remain the same - set as in non-UTF mode, all with values less than 256. This remains - true even when PCRE is built to include Unicode property support, - because to do otherwise would slow down PCRE in many common cases. Note - in particular that this applies to \b and \B, because they are defined - in terms of \w and \W. If you really want to test for a wider sense of, - say, "digit", you can use explicit Unicode property tests such as - \p{Nd}. Alternatively, if you set the PCRE_UCP option, the way that the - character escapes work is changed so that Unicode properties are used - to determine which characters match. There are more details in the sec- - tion on generic character types in the pcrepattern documentation. - - 7. Similarly, characters that match the POSIX named character classes - are all low-valued characters, unless the PCRE_UCP option is set. - - 8. However, the horizontal and vertical white space matching escapes - (\h, \H, \v, and \V) do match all the appropriate Unicode characters, - whether or not PCRE_UCP is set. - - 9. Case-insensitive matching applies only to characters whose values - are less than 128, unless PCRE is built with Unicode property support. - A few Unicode characters such as Greek sigma have more than two code- - points that are case-equivalent. Up to and including PCRE release 8.31, - only one-to-one case mappings were supported, but later releases (with - Unicode property support) do treat as case-equivalent all versions of - characters such as Greek sigma. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 27 February 2013 - Copyright (c) 1997-2013 University of Cambridge. ------------------------------------------------------------------------------- - - -PCREJIT(3) Library Functions Manual PCREJIT(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -PCRE JUST-IN-TIME COMPILER SUPPORT - - Just-in-time compiling is a heavyweight optimization that can greatly - speed up pattern matching. However, it comes at the cost of extra pro- - cessing before the match is performed. Therefore, it is of most benefit - when the same pattern is going to be matched many times. This does not - necessarily mean many calls of a matching function; if the pattern is - not anchored, matching attempts may take place many times at various - positions in the subject, even for a single call. Therefore, if the - subject string is very long, it may still pay to use JIT for one-off - matches. - - JIT support applies only to the traditional Perl-compatible matching - function. It does not apply when the DFA matching function is being - used. The code for this support was written by Zoltan Herczeg. - - -8-BIT, 16-BIT AND 32-BIT SUPPORT - - JIT support is available for all of the 8-bit, 16-bit and 32-bit PCRE - libraries. To keep this documentation simple, only the 8-bit interface - is described in what follows. If you are using the 16-bit library, sub- - stitute the 16-bit functions and 16-bit structures (for example, - pcre16_jit_stack instead of pcre_jit_stack). If you are using the - 32-bit library, substitute the 32-bit functions and 32-bit structures - (for example, pcre32_jit_stack instead of pcre_jit_stack). - - -AVAILABILITY OF JIT SUPPORT - - JIT support is an optional feature of PCRE. The "configure" option - --enable-jit (or equivalent CMake option) must be set when PCRE is - built if you want to use JIT. The support is limited to the following - hardware platforms: - - ARM v5, v7, and Thumb2 - Intel x86 32-bit and 64-bit - MIPS 32-bit - Power PC 32-bit and 64-bit - SPARC 32-bit (experimental) - - If --enable-jit is set on an unsupported platform, compilation fails. - - A program that is linked with PCRE 8.20 or later can tell if JIT sup- - port is available by calling pcre_config() with the PCRE_CONFIG_JIT - option. The result is 1 when JIT is available, and 0 otherwise. How- - ever, a simple program does not need to check this in order to use JIT. - The normal API is implemented in a way that falls back to the interpre- - tive code if JIT is not available. For programs that need the best pos- - sible performance, there is also a "fast path" API that is JIT-spe- - cific. - - If your program may sometimes be linked with versions of PCRE that are - older than 8.20, but you want to use JIT when it is available, you can - test the values of PCRE_MAJOR and PCRE_MINOR, or the existence of a JIT - macro such as PCRE_CONFIG_JIT, for compile-time control of your code. - - -SIMPLE USE OF JIT - - You have to do two things to make use of the JIT support in the sim- - plest way: - - (1) Call pcre_study() with the PCRE_STUDY_JIT_COMPILE option for - each compiled pattern, and pass the resulting pcre_extra block to - pcre_exec(). - - (2) Use pcre_free_study() to free the pcre_extra block when it is - no longer needed, instead of just freeing it yourself. This - ensures that - any JIT data is also freed. - - For a program that may be linked with pre-8.20 versions of PCRE, you - can insert - - #ifndef PCRE_STUDY_JIT_COMPILE - #define PCRE_STUDY_JIT_COMPILE 0 - #endif - - so that no option is passed to pcre_study(), and then use something - like this to free the study data: - - #ifdef PCRE_CONFIG_JIT - pcre_free_study(study_ptr); - #else - pcre_free(study_ptr); - #endif - - PCRE_STUDY_JIT_COMPILE requests the JIT compiler to generate code for - complete matches. If you want to run partial matches using the - PCRE_PARTIAL_HARD or PCRE_PARTIAL_SOFT options of pcre_exec(), you - should set one or both of the following options in addition to, or - instead of, PCRE_STUDY_JIT_COMPILE when you call pcre_study(): - - PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE - PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE - - The JIT compiler generates different optimized code for each of the - three modes (normal, soft partial, hard partial). When pcre_exec() is - called, the appropriate code is run if it is available. Otherwise, the - pattern is matched using interpretive code. - - In some circumstances you may need to call additional functions. These - are described in the section entitled "Controlling the JIT stack" - below. - - If JIT support is not available, PCRE_STUDY_JIT_COMPILE etc. are - ignored, and no JIT data is created. Otherwise, the compiled pattern is - passed to the JIT compiler, which turns it into machine code that exe- - cutes much faster than the normal interpretive code. When pcre_exec() - is passed a pcre_extra block containing a pointer to JIT code of the - appropriate mode (normal or hard/soft partial), it obeys that code - instead of running the interpreter. The result is identical, but the - compiled JIT code runs much faster. - - There are some pcre_exec() options that are not supported for JIT exe- - cution. There are also some pattern items that JIT cannot handle. - Details are given below. In both cases, execution automatically falls - back to the interpretive code. If you want to know whether JIT was - actually used for a particular match, you should arrange for a JIT - callback function to be set up as described in the section entitled - "Controlling the JIT stack" below, even if you do not need to supply a - non-default JIT stack. Such a callback function is called whenever JIT - code is about to be obeyed. If the execution options are not right for - JIT execution, the callback function is not obeyed. - - If the JIT compiler finds an unsupported item, no JIT data is gener- - ated. You can find out if JIT execution is available after studying a - pattern by calling pcre_fullinfo() with the PCRE_INFO_JIT option. A - result of 1 means that JIT compilation was successful. A result of 0 - means that JIT support is not available, or the pattern was not studied - with PCRE_STUDY_JIT_COMPILE etc., or the JIT compiler was not able to - handle the pattern. - - Once a pattern has been studied, with or without JIT, it can be used as - many times as you like for matching different subject strings. - - -UNSUPPORTED OPTIONS AND PATTERN ITEMS - - The only pcre_exec() options that are supported for JIT execution are - PCRE_NO_UTF8_CHECK, PCRE_NO_UTF16_CHECK, PCRE_NO_UTF32_CHECK, PCRE_NOT- - BOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, PCRE_PAR- - TIAL_HARD, and PCRE_PARTIAL_SOFT. - - The only unsupported pattern items are \C (match a single data unit) - when running in a UTF mode, and a callout immediately before an asser- - tion condition in a conditional group. - - -RETURN VALUES FROM JIT EXECUTION - - When a pattern is matched using JIT execution, the return values are - the same as those given by the interpretive pcre_exec() code, with the - addition of one new error code: PCRE_ERROR_JIT_STACKLIMIT. This means - that the memory used for the JIT stack was insufficient. See "Control- - ling the JIT stack" below for a discussion of JIT stack usage. For com- - patibility with the interpretive pcre_exec() code, no more than two- - thirds of the ovector argument is used for passing back captured sub- - strings. - - The error code PCRE_ERROR_MATCHLIMIT is returned by the JIT code if - searching a very large pattern tree goes on for too long, as it is in - the same circumstance when JIT is not used, but the details of exactly - what is counted are not the same. The PCRE_ERROR_RECURSIONLIMIT error - code is never returned by JIT execution. - - -SAVING AND RESTORING COMPILED PATTERNS - - The code that is generated by the JIT compiler is architecture-spe- - cific, and is also position dependent. For those reasons it cannot be - saved (in a file or database) and restored later like the bytecode and - other data of a compiled pattern. Saving and restoring compiled pat- - terns is not something many people do. More detail about this facility - is given in the pcreprecompile documentation. It should be possible to - run pcre_study() on a saved and restored pattern, and thereby recreate - the JIT data, but because JIT compilation uses significant resources, - it is probably not worth doing this; you might as well recompile the - original pattern. - - -CONTROLLING THE JIT STACK - - When the compiled JIT code runs, it needs a block of memory to use as a - stack. By default, it uses 32K on the machine stack. However, some - large or complicated patterns need more than this. The error - PCRE_ERROR_JIT_STACKLIMIT is given when there is not enough stack. - Three functions are provided for managing blocks of memory for use as - JIT stacks. There is further discussion about the use of JIT stacks in - the section entitled "JIT stack FAQ" below. - - The pcre_jit_stack_alloc() function creates a JIT stack. Its arguments - are a starting size and a maximum size, and it returns a pointer to an - opaque structure of type pcre_jit_stack, or NULL if there is an error. - The pcre_jit_stack_free() function can be used to free a stack that is - no longer needed. (For the technically minded: the address space is - allocated by mmap or VirtualAlloc.) - - JIT uses far less memory for recursion than the interpretive code, and - a maximum stack size of 512K to 1M should be more than enough for any - pattern. - - The pcre_assign_jit_stack() function specifies which stack JIT code - should use. Its arguments are as follows: - - pcre_extra *extra - pcre_jit_callback callback - void *data - - The extra argument must be the result of studying a pattern with - PCRE_STUDY_JIT_COMPILE etc. There are three cases for the values of the - other two options: - - (1) If callback is NULL and data is NULL, an internal 32K block - on the machine stack is used. - - (2) If callback is NULL and data is not NULL, data must be - a valid JIT stack, the result of calling pcre_jit_stack_alloc(). - - (3) If callback is not NULL, it must point to a function that is - called with data as an argument at the start of matching, in - order to set up a JIT stack. If the return from the callback - function is NULL, the internal 32K stack is used; otherwise the - return value must be a valid JIT stack, the result of calling - pcre_jit_stack_alloc(). - - A callback function is obeyed whenever JIT code is about to be run; it - is not obeyed when pcre_exec() is called with options that are incom- - patible for JIT execution. A callback function can therefore be used to - determine whether a match operation was executed by JIT or by the - interpreter. - - You may safely use the same JIT stack for more than one pattern (either - by assigning directly or by callback), as long as the patterns are all - matched sequentially in the same thread. In a multithread application, - if you do not specify a JIT stack, or if you assign or pass back NULL - from a callback, that is thread-safe, because each thread has its own - machine stack. However, if you assign or pass back a non-NULL JIT - stack, this must be a different stack for each thread so that the - application is thread-safe. - - Strictly speaking, even more is allowed. You can assign the same non- - NULL stack to any number of patterns as long as they are not used for - matching by multiple threads at the same time. For example, you can - assign the same stack to all compiled patterns, and use a global mutex - in the callback to wait until the stack is available for use. However, - this is an inefficient solution, and not recommended. - - This is a suggestion for how a multithreaded program that needs to set - up non-default JIT stacks might operate: - - During thread initalization - thread_local_var = pcre_jit_stack_alloc(...) - - During thread exit - pcre_jit_stack_free(thread_local_var) - - Use a one-line callback function - return thread_local_var - - All the functions described in this section do nothing if JIT is not - available, and pcre_assign_jit_stack() does nothing unless the extra - argument is non-NULL and points to a pcre_extra block that is the - result of a successful study with PCRE_STUDY_JIT_COMPILE etc. - - -JIT STACK FAQ - - (1) Why do we need JIT stacks? - - PCRE (and JIT) is a recursive, depth-first engine, so it needs a stack - where the local data of the current node is pushed before checking its - child nodes. Allocating real machine stack on some platforms is diffi- - cult. For example, the stack chain needs to be updated every time if we - extend the stack on PowerPC. Although it is possible, its updating - time overhead decreases performance. So we do the recursion in memory. - - (2) Why don't we simply allocate blocks of memory with malloc()? - - Modern operating systems have a nice feature: they can reserve an - address space instead of allocating memory. We can safely allocate mem- - ory pages inside this address space, so the stack could grow without - moving memory data (this is important because of pointers). Thus we can - allocate 1M address space, and use only a single memory page (usually - 4K) if that is enough. However, we can still grow up to 1M anytime if - needed. - - (3) Who "owns" a JIT stack? - - The owner of the stack is the user program, not the JIT studied pattern - or anything else. The user program must ensure that if a stack is used - by pcre_exec(), (that is, it is assigned to the pattern currently run- - ning), that stack must not be used by any other threads (to avoid over- - writing the same memory area). The best practice for multithreaded pro- - grams is to allocate a stack for each thread, and return this stack - through the JIT callback function. - - (4) When should a JIT stack be freed? - - You can free a JIT stack at any time, as long as it will not be used by - pcre_exec() again. When you assign the stack to a pattern, only a - pointer is set. There is no reference counting or any other magic. You - can free the patterns and stacks in any order, anytime. Just do not - call pcre_exec() with a pattern pointing to an already freed stack, as - that will cause SEGFAULT. (Also, do not free a stack currently used by - pcre_exec() in another thread). You can also replace the stack for a - pattern at any time. You can even free the previous stack before - assigning a replacement. - - (5) Should I allocate/free a stack every time before/after calling - pcre_exec()? - - No, because this is too costly in terms of resources. However, you - could implement some clever idea which release the stack if it is not - used in let's say two minutes. The JIT callback can help to achieve - this without keeping a list of the currently JIT studied patterns. - - (6) OK, the stack is for long term memory allocation. But what happens - if a pattern causes stack overflow with a stack of 1M? Is that 1M kept - until the stack is freed? - - Especially on embedded sytems, it might be a good idea to release mem- - ory sometimes without freeing the stack. There is no API for this at - the moment. Probably a function call which returns with the currently - allocated memory for any stack and another which allows releasing mem- - ory (shrinking the stack) would be a good idea if someone needs this. - - (7) This is too much of a headache. Isn't there any better solution for - JIT stack handling? - - No, thanks to Windows. If POSIX threads were used everywhere, we could - throw out this complicated API. - - -EXAMPLE CODE - - This is a single-threaded example that specifies a JIT stack without - using a callback. - - int rc; - int ovector[30]; - pcre *re; - pcre_extra *extra; - pcre_jit_stack *jit_stack; - - re = pcre_compile(pattern, 0, &error, &erroffset, NULL); - /* Check for errors */ - extra = pcre_study(re, PCRE_STUDY_JIT_COMPILE, &error); - jit_stack = pcre_jit_stack_alloc(32*1024, 512*1024); - /* Check for error (NULL) */ - pcre_assign_jit_stack(extra, NULL, jit_stack); - rc = pcre_exec(re, extra, subject, length, 0, 0, ovector, 30); - /* Check results */ - pcre_free(re); - pcre_free_study(extra); - pcre_jit_stack_free(jit_stack); - - -JIT FAST PATH API - - Because the API described above falls back to interpreted execution - when JIT is not available, it is convenient for programs that are writ- - ten for general use in many environments. However, calling JIT via - pcre_exec() does have a performance impact. Programs that are written - for use where JIT is known to be available, and which need the best - possible performance, can instead use a "fast path" API to call JIT - execution directly instead of calling pcre_exec() (obviously only for - patterns that have been successfully studied by JIT). - - The fast path function is called pcre_jit_exec(), and it takes exactly - the same arguments as pcre_exec(), plus one additional argument that - must point to a JIT stack. The JIT stack arrangements described above - do not apply. The return values are the same as for pcre_exec(). - - When you call pcre_exec(), as well as testing for invalid options, a - number of other sanity checks are performed on the arguments. For exam- - ple, if the subject pointer is NULL, or its length is negative, an - immediate error is given. Also, unless PCRE_NO_UTF[8|16|32] is set, a - UTF subject string is tested for validity. In the interests of speed, - these checks do not happen on the JIT fast path, and if invalid data is - passed, the result is undefined. - - Bypassing the sanity checks and the pcre_exec() wrapping can give - speedups of more than 10%. - - -SEE ALSO - - pcreapi(3) - - -AUTHOR - - Philip Hazel (FAQ by Zoltan Herczeg) - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 17 March 2013 - Copyright (c) 1997-2013 University of Cambridge. ------------------------------------------------------------------------------- - - -PCREPARTIAL(3) Library Functions Manual PCREPARTIAL(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -PARTIAL MATCHING IN PCRE - - In normal use of PCRE, if the subject string that is passed to a match- - ing function matches as far as it goes, but is too short to match the - entire pattern, PCRE_ERROR_NOMATCH is returned. There are circumstances - where it might be helpful to distinguish this case from other cases in - which there is no match. - - Consider, for example, an application where a human is required to type - in data for a field with specific formatting requirements. An example - might be a date in the form ddmmmyy, defined by this pattern: - - ^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$ - - If the application sees the user's keystrokes one by one, and can check - that what has been typed so far is potentially valid, it is able to - raise an error as soon as a mistake is made, by beeping and not - reflecting the character that has been typed, for example. This immedi- - ate feedback is likely to be a better user interface than a check that - is delayed until the entire string has been entered. Partial matching - can also be useful when the subject string is very long and is not all - available at once. - - PCRE supports partial matching by means of the PCRE_PARTIAL_SOFT and - PCRE_PARTIAL_HARD options, which can be set when calling any of the - matching functions. For backwards compatibility, PCRE_PARTIAL is a syn- - onym for PCRE_PARTIAL_SOFT. The essential difference between the two - options is whether or not a partial match is preferred to an alterna- - tive complete match, though the details differ between the two types of - matching function. If both options are set, PCRE_PARTIAL_HARD takes - precedence. - - If you want to use partial matching with just-in-time optimized code, - you must call pcre_study(), pcre16_study() or pcre32_study() with one - or both of these options: - - PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE - PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE - - PCRE_STUDY_JIT_COMPILE should also be set if you are going to run non- - partial matches on the same pattern. If the appropriate JIT study mode - has not been set for a match, the interpretive matching code is used. - - Setting a partial matching option disables two of PCRE's standard opti- - mizations. PCRE remembers the last literal data unit in a pattern, and - abandons matching immediately if it is not present in the subject - string. This optimization cannot be used for a subject string that - might match only partially. If the pattern was studied, PCRE knows the - minimum length of a matching string, and does not bother to run the - matching function on shorter strings. This optimization is also dis- - abled for partial matching. - - -PARTIAL MATCHING USING pcre_exec() OR pcre[16|32]_exec() - - A partial match occurs during a call to pcre_exec() or - pcre[16|32]_exec() when the end of the subject string is reached suc- - cessfully, but matching cannot continue because more characters are - needed. However, at least one character in the subject must have been - inspected. This character need not form part of the final matched - string; lookbehind assertions and the \K escape sequence provide ways - of inspecting characters before the start of a matched substring. The - requirement for inspecting at least one character exists because an - empty string can always be matched; without such a restriction there - would always be a partial match of an empty string at the end of the - subject. - - If there are at least two slots in the offsets vector when a partial - match is returned, the first slot is set to the offset of the earliest - character that was inspected. For convenience, the second offset points - to the end of the subject so that a substring can easily be identified. - If there are at least three slots in the offsets vector, the third slot - is set to the offset of the character where matching started. - - For the majority of patterns, the contents of the first and third slots - will be the same. However, for patterns that contain lookbehind asser- - tions, or begin with \b or \B, characters before the one where matching - started may have been inspected while carrying out the match. For exam- - ple, consider this pattern: - - /(?<=abc)123/ - - This pattern matches "123", but only if it is preceded by "abc". If the - subject string is "xyzabc12", the first two offsets after a partial - match are for the substring "abc12", because all these characters were - inspected. However, the third offset is set to 6, because that is the - offset where matching began. - - What happens when a partial match is identified depends on which of the - two partial matching options are set. - - PCRE_PARTIAL_SOFT WITH pcre_exec() OR pcre[16|32]_exec() - - If PCRE_PARTIAL_SOFT is set when pcre_exec() or pcre[16|32]_exec() - identifies a partial match, the partial match is remembered, but match- - ing continues as normal, and other alternatives in the pattern are - tried. If no complete match can be found, PCRE_ERROR_PARTIAL is - returned instead of PCRE_ERROR_NOMATCH. - - This option is "soft" because it prefers a complete match over a par- - tial match. All the various matching items in a pattern behave as if - the subject string is potentially complete. For example, \z, \Z, and $ - match at the end of the subject, as normal, and for \b and \B the end - of the subject is treated as a non-alphanumeric. - - If there is more than one partial match, the first one that was found - provides the data that is returned. Consider this pattern: - - /123\w+X|dogY/ - - If this is matched against the subject string "abc123dog", both alter- - natives fail to match, but the end of the subject is reached during - matching, so PCRE_ERROR_PARTIAL is returned. The offsets are set to 3 - and 9, identifying "123dog" as the first partial match that was found. - (In this example, there are two partial matches, because "dog" on its - own partially matches the second alternative.) - - PCRE_PARTIAL_HARD WITH pcre_exec() OR pcre[16|32]_exec() - - If PCRE_PARTIAL_HARD is set for pcre_exec() or pcre[16|32]_exec(), - PCRE_ERROR_PARTIAL is returned as soon as a partial match is found, - without continuing to search for possible complete matches. This option - is "hard" because it prefers an earlier partial match over a later com- - plete match. For this reason, the assumption is made that the end of - the supplied subject string may not be the true end of the available - data, and so, if \z, \Z, \b, \B, or $ are encountered at the end of the - subject, the result is PCRE_ERROR_PARTIAL, provided that at least one - character in the subject has been inspected. - - Setting PCRE_PARTIAL_HARD also affects the way UTF-8 and UTF-16 subject - strings are checked for validity. Normally, an invalid sequence causes - the error PCRE_ERROR_BADUTF8 or PCRE_ERROR_BADUTF16. However, in the - special case of a truncated character at the end of the subject, - PCRE_ERROR_SHORTUTF8 or PCRE_ERROR_SHORTUTF16 is returned when - PCRE_PARTIAL_HARD is set. - - Comparing hard and soft partial matching - - The difference between the two partial matching options can be illus- - trated by a pattern such as: - - /dog(sbody)?/ - - This matches either "dog" or "dogsbody", greedily (that is, it prefers - the longer string if possible). If it is matched against the string - "dog" with PCRE_PARTIAL_SOFT, it yields a complete match for "dog". - However, if PCRE_PARTIAL_HARD is set, the result is PCRE_ERROR_PARTIAL. - On the other hand, if the pattern is made ungreedy the result is dif- - ferent: - - /dog(sbody)??/ - - In this case the result is always a complete match because that is - found first, and matching never continues after finding a complete - match. It might be easier to follow this explanation by thinking of the - two patterns like this: - - /dog(sbody)?/ is the same as /dogsbody|dog/ - /dog(sbody)??/ is the same as /dog|dogsbody/ - - The second pattern will never match "dogsbody", because it will always - find the shorter match first. - - -PARTIAL MATCHING USING pcre_dfa_exec() OR pcre[16|32]_dfa_exec() - - The DFA functions move along the subject string character by character, - without backtracking, searching for all possible matches simultane- - ously. If the end of the subject is reached before the end of the pat- - tern, there is the possibility of a partial match, again provided that - at least one character has been inspected. - - When PCRE_PARTIAL_SOFT is set, PCRE_ERROR_PARTIAL is returned only if - there have been no complete matches. Otherwise, the complete matches - are returned. However, if PCRE_PARTIAL_HARD is set, a partial match - takes precedence over any complete matches. The portion of the string - that was inspected when the longest partial match was found is set as - the first matching string, provided there are at least two slots in the - offsets vector. - - Because the DFA functions always search for all possible matches, and - there is no difference between greedy and ungreedy repetition, their - behaviour is different from the standard functions when PCRE_PAR- - TIAL_HARD is set. Consider the string "dog" matched against the - ungreedy pattern shown above: - - /dog(sbody)??/ - - Whereas the standard functions stop as soon as they find the complete - match for "dog", the DFA functions also find the partial match for - "dogsbody", and so return that when PCRE_PARTIAL_HARD is set. - - -PARTIAL MATCHING AND WORD BOUNDARIES - - If a pattern ends with one of sequences \b or \B, which test for word - boundaries, partial matching with PCRE_PARTIAL_SOFT can give counter- - intuitive results. Consider this pattern: - - /\bcat\b/ - - This matches "cat", provided there is a word boundary at either end. If - the subject string is "the cat", the comparison of the final "t" with a - following character cannot take place, so a partial match is found. - However, normal matching carries on, and \b matches at the end of the - subject when the last character is a letter, so a complete match is - found. The result, therefore, is not PCRE_ERROR_PARTIAL. Using - PCRE_PARTIAL_HARD in this case does yield PCRE_ERROR_PARTIAL, because - then the partial match takes precedence. - - -FORMERLY RESTRICTED PATTERNS - - For releases of PCRE prior to 8.00, because of the way certain internal - optimizations were implemented in the pcre_exec() function, the - PCRE_PARTIAL option (predecessor of PCRE_PARTIAL_SOFT) could not be - used with all patterns. From release 8.00 onwards, the restrictions no - longer apply, and partial matching with can be requested for any pat- - tern. - - Items that were formerly restricted were repeated single characters and - repeated metasequences. If PCRE_PARTIAL was set for a pattern that did - not conform to the restrictions, pcre_exec() returned the error code - PCRE_ERROR_BADPARTIAL (-13). This error code is no longer in use. The - PCRE_INFO_OKPARTIAL call to pcre_fullinfo() to find out if a compiled - pattern can be used for partial matching now always returns 1. - - -EXAMPLE OF PARTIAL MATCHING USING PCRETEST - - If the escape sequence \P is present in a pcretest data line, the - PCRE_PARTIAL_SOFT option is used for the match. Here is a run of - pcretest that uses the date example quoted above: - - re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ - data> 25jun04\P - 0: 25jun04 - 1: jun - data> 25dec3\P - Partial match: 23dec3 - data> 3ju\P - Partial match: 3ju - data> 3juj\P - No match - data> j\P - No match - - The first data string is matched completely, so pcretest shows the - matched substrings. The remaining four strings do not match the com- - plete pattern, but the first two are partial matches. Similar output is - obtained if DFA matching is used. - - If the escape sequence \P is present more than once in a pcretest data - line, the PCRE_PARTIAL_HARD option is set for the match. - - -MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre[16|32]_dfa_exec() - - When a partial match has been found using a DFA matching function, it - is possible to continue the match by providing additional subject data - and calling the function again with the same compiled regular expres- - sion, this time setting the PCRE_DFA_RESTART option. You must pass the - same working space as before, because this is where details of the pre- - vious partial match are stored. Here is an example using pcretest, - using the \R escape sequence to set the PCRE_DFA_RESTART option (\D - specifies the use of the DFA matching function): - - re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ - data> 23ja\P\D - Partial match: 23ja - data> n05\R\D - 0: n05 - - The first call has "23ja" as the subject, and requests partial match- - ing; the second call has "n05" as the subject for the continued - (restarted) match. Notice that when the match is complete, only the - last part is shown; PCRE does not retain the previously partially- - matched string. It is up to the calling program to do that if it needs - to. - - That means that, for an unanchored pattern, if a continued match fails, - it is not possible to try again at a new starting point. All this - facility is capable of doing is continuing with the previous match - attempt. In the previous example, if the second set of data is "ug23" - the result is no match, even though there would be a match for "aug23" - if the entire string were given at once. Depending on the application, - this may or may not be what you want. The only way to allow for start- - ing again at the next character is to retain the matched part of the - subject and try a new complete match. - - You can set the PCRE_PARTIAL_SOFT or PCRE_PARTIAL_HARD options with - PCRE_DFA_RESTART to continue partial matching over multiple segments. - This facility can be used to pass very long subject strings to the DFA - matching functions. - - -MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre[16|32]_exec() - - From release 8.00, the standard matching functions can also be used to - do multi-segment matching. Unlike the DFA functions, it is not possible - to restart the previous match with a new segment of data. Instead, new - data must be added to the previous subject string, and the entire match - re-run, starting from the point where the partial match occurred. Ear- - lier data can be discarded. - - It is best to use PCRE_PARTIAL_HARD in this situation, because it does - not treat the end of a segment as the end of the subject when matching - \z, \Z, \b, \B, and $. Consider an unanchored pattern that matches - dates: - - re> /\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d/ - data> The date is 23ja\P\P - Partial match: 23ja - - At this stage, an application could discard the text preceding "23ja", - add on text from the next segment, and call the matching function - again. Unlike the DFA matching functions, the entire matching string - must always be available, and the complete matching process occurs for - each call, so more memory and more processing time is needed. - - Note: If the pattern contains lookbehind assertions, or \K, or starts - with \b or \B, the string that is returned for a partial match includes - characters that precede the start of what would be returned for a com- - plete match, because it contains all the characters that were inspected - during the partial match. - - -ISSUES WITH MULTI-SEGMENT MATCHING - - Certain types of pattern may give problems with multi-segment matching, - whichever matching function is used. - - 1. If the pattern contains a test for the beginning of a line, you need - to pass the PCRE_NOTBOL option when the subject string for any call - does start at the beginning of a line. There is also a PCRE_NOTEOL - option, but in practice when doing multi-segment matching you should be - using PCRE_PARTIAL_HARD, which includes the effect of PCRE_NOTEOL. - - 2. Lookbehind assertions that have already been obeyed are catered for - in the offsets that are returned for a partial match. However a lookbe- - hind assertion later in the pattern could require even earlier charac- - ters to be inspected. You can handle this case by using the - PCRE_INFO_MAXLOOKBEHIND option of the pcre_fullinfo() or - pcre[16|32]_fullinfo() functions to obtain the length of the longest - lookbehind in the pattern. This length is given in characters, not - bytes. If you always retain at least that many characters before the - partially matched string, all should be well. (Of course, near the - start of the subject, fewer characters may be present; in that case all - characters should be retained.) - - From release 8.33, there is a more accurate way of deciding which char- - acters to retain. Instead of subtracting the length of the longest - lookbehind from the earliest inspected character (offsets[0]), the - match start position (offsets[2]) should be used, and the next match - attempt started at the offsets[2] character by setting the startoffset - argument of pcre_exec() or pcre_dfa_exec(). - - For example, if the pattern "(?<=123)abc" is partially matched against - the string "xx123a", the three offset values returned are 2, 6, and 5. - This indicates that the matching process that gave a partial match - started at offset 5, but the characters "123a" were all inspected. The - maximum lookbehind for that pattern is 3, so taking that away from 5 - shows that we need only keep "123a", and the next match attempt can be - started at offset 3 (that is, at "a") when further characters have been - added. When the match start is not the earliest inspected character, - pcretest shows it explicitly: - - re> "(?<=123)abc" - data> xx123a\P\P - Partial match at offset 5: 123a - - 3. Because a partial match must always contain at least one character, - what might be considered a partial match of an empty string actually - gives a "no match" result. For example: - - re> /c(?<=abc)x/ - data> ab\P - No match - - If the next segment begins "cx", a match should be found, but this will - only happen if characters from the previous segment are retained. For - this reason, a "no match" result should be interpreted as "partial - match of an empty string" when the pattern contains lookbehinds. - - 4. Matching a subject string that is split into multiple segments may - not always produce exactly the same result as matching over one single - long string, especially when PCRE_PARTIAL_SOFT is used. The section - "Partial Matching and Word Boundaries" above describes an issue that - arises if the pattern ends with \b or \B. Another kind of difference - may occur when there are multiple matching possibilities, because (for - PCRE_PARTIAL_SOFT) a partial match result is given only when there are - no completed matches. This means that as soon as the shortest match has - been found, continuation to a new subject segment is no longer possi- - ble. Consider again this pcretest example: - - re> /dog(sbody)?/ - data> dogsb\P - 0: dog - data> do\P\D - Partial match: do - data> gsb\R\P\D - 0: g - data> dogsbody\D - 0: dogsbody - 1: dog - - The first data line passes the string "dogsb" to a standard matching - function, setting the PCRE_PARTIAL_SOFT option. Although the string is - a partial match for "dogsbody", the result is not PCRE_ERROR_PARTIAL, - because the shorter string "dog" is a complete match. Similarly, when - the subject is presented to a DFA matching function in several parts - ("do" and "gsb" being the first two) the match stops when "dog" has - been found, and it is not possible to continue. On the other hand, if - "dogsbody" is presented as a single string, a DFA matching function - finds both matches. - - Because of these problems, it is best to use PCRE_PARTIAL_HARD when - matching multi-segment data. The example above then behaves differ- - ently: - - re> /dog(sbody)?/ - data> dogsb\P\P - Partial match: dogsb - data> do\P\D - Partial match: do - data> gsb\R\P\P\D - Partial match: gsb - - 5. Patterns that contain alternatives at the top level which do not all - start with the same pattern item may not work as expected when - PCRE_DFA_RESTART is used. For example, consider this pattern: - - 1234|3789 - - If the first part of the subject is "ABC123", a partial match of the - first alternative is found at offset 3. There is no partial match for - the second alternative, because such a match does not start at the same - point in the subject string. Attempting to continue with the string - "7890" does not yield a match because only those alternatives that - match at one point in the subject are remembered. The problem arises - because the start of the second alternative matches within the first - alternative. There is no problem with anchored patterns or patterns - such as: - - 1234|ABCD - - where no string can be a partial match for both alternatives. This is - not a problem if a standard matching function is used, because the - entire match has to be rerun each time: - - re> /1234|3789/ - data> ABC123\P\P - Partial match: 123 - data> 1237890 - 0: 3789 - - Of course, instead of using PCRE_DFA_RESTART, the same technique of re- - running the entire match can also be used with the DFA matching func- - tions. Another possibility is to work with two buffers. If a partial - match at offset n in the first buffer is followed by "no match" when - PCRE_DFA_RESTART is used on the second buffer, you can then try a new - match starting at offset n+1 in the first buffer. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 02 July 2013 - Copyright (c) 1997-2013 University of Cambridge. ------------------------------------------------------------------------------- - - -PCREPRECOMPILE(3) Library Functions Manual PCREPRECOMPILE(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -SAVING AND RE-USING PRECOMPILED PCRE PATTERNS - - If you are running an application that uses a large number of regular - expression patterns, it may be useful to store them in a precompiled - form instead of having to compile them every time the application is - run. If you are not using any private character tables (see the - pcre_maketables() documentation), this is relatively straightforward. - If you are using private tables, it is a little bit more complicated. - However, if you are using the just-in-time optimization feature, it is - not possible to save and reload the JIT data. - - If you save compiled patterns to a file, you can copy them to a differ- - ent host and run them there. If the two hosts have different endianness - (byte order), you should run the pcre[16|32]_pat- - tern_to_host_byte_order() function on the new host before trying to - match the pattern. The matching functions return PCRE_ERROR_BADENDIAN- - NESS if they detect a pattern with the wrong endianness. - - Compiling regular expressions with one version of PCRE for use with a - different version is not guaranteed to work and may cause crashes, and - saving and restoring a compiled pattern loses any JIT optimization - data. - - -SAVING A COMPILED PATTERN - - The value returned by pcre[16|32]_compile() points to a single block of - memory that holds the compiled pattern and associated data. You can - find the length of this block in bytes by calling - pcre[16|32]_fullinfo() with an argument of PCRE_INFO_SIZE. You can then - save the data in any appropriate manner. Here is sample code for the - 8-bit library that compiles a pattern and writes it to a file. It - assumes that the variable fd refers to a file that is open for output: - - int erroroffset, rc, size; - char *error; - pcre *re; - - re = pcre_compile("my pattern", 0, &error, &erroroffset, NULL); - if (re == NULL) { ... handle errors ... } - rc = pcre_fullinfo(re, NULL, PCRE_INFO_SIZE, &size); - if (rc < 0) { ... handle errors ... } - rc = fwrite(re, 1, size, fd); - if (rc != size) { ... handle errors ... } - - In this example, the bytes that comprise the compiled pattern are - copied exactly. Note that this is binary data that may contain any of - the 256 possible byte values. On systems that make a distinction - between binary and non-binary data, be sure that the file is opened for - binary output. - - If you want to write more than one pattern to a file, you will have to - devise a way of separating them. For binary data, preceding each pat- - tern with its length is probably the most straightforward approach. - Another possibility is to write out the data in hexadecimal instead of - binary, one pattern to a line. - - Saving compiled patterns in a file is only one possible way of storing - them for later use. They could equally well be saved in a database, or - in the memory of some daemon process that passes them via sockets to - the processes that want them. - - If the pattern has been studied, it is also possible to save the normal - study data in a similar way to the compiled pattern itself. However, if - the PCRE_STUDY_JIT_COMPILE was used, the just-in-time data that is cre- - ated cannot be saved because it is too dependent on the current envi- - ronment. When studying generates additional information, - pcre[16|32]_study() returns a pointer to a pcre[16|32]_extra data - block. Its format is defined in the section on matching a pattern in - the pcreapi documentation. The study_data field points to the binary - study data, and this is what you must save (not the pcre[16|32]_extra - block itself). The length of the study data can be obtained by calling - pcre[16|32]_fullinfo() with an argument of PCRE_INFO_STUDYSIZE. Remem- - ber to check that pcre[16|32]_study() did return a non-NULL value - before trying to save the study data. - - -RE-USING A PRECOMPILED PATTERN - - Re-using a precompiled pattern is straightforward. Having reloaded it - into main memory, called pcre[16|32]_pattern_to_host_byte_order() if - necessary, you pass its pointer to pcre[16|32]_exec() or - pcre[16|32]_dfa_exec() in the usual way. - - However, if you passed a pointer to custom character tables when the - pattern was compiled (the tableptr argument of pcre[16|32]_compile()), - you must now pass a similar pointer to pcre[16|32]_exec() or - pcre[16|32]_dfa_exec(), because the value saved with the compiled pat- - tern will obviously be nonsense. A field in a pcre[16|32]_extra() block - is used to pass this data, as described in the section on matching a - pattern in the pcreapi documentation. - - Warning: The tables that pcre_exec() and pcre_dfa_exec() use must be - the same as those that were used when the pattern was compiled. If this - is not the case, the behaviour is undefined. - - If you did not provide custom character tables when the pattern was - compiled, the pointer in the compiled pattern is NULL, which causes the - matching functions to use PCRE's internal tables. Thus, you do not need - to take any special action at run time in this case. - - If you saved study data with the compiled pattern, you need to create - your own pcre[16|32]_extra data block and set the study_data field to - point to the reloaded study data. You must also set the - PCRE_EXTRA_STUDY_DATA bit in the flags field to indicate that study - data is present. Then pass the pcre[16|32]_extra block to the matching - function in the usual way. If the pattern was studied for just-in-time - optimization, that data cannot be saved, and so is lost by a - save/restore cycle. - - -COMPATIBILITY WITH DIFFERENT PCRE RELEASES - - In general, it is safest to recompile all saved patterns when you - update to a new PCRE release, though not all updates actually require - this. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 12 November 2013 - Copyright (c) 1997-2013 University of Cambridge. ------------------------------------------------------------------------------- - - -PCREPERFORM(3) Library Functions Manual PCREPERFORM(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -PCRE PERFORMANCE - - Two aspects of performance are discussed below: memory usage and pro- - cessing time. The way you express your pattern as a regular expression - can affect both of them. - - -COMPILED PATTERN MEMORY USAGE - - Patterns are compiled by PCRE into a reasonably efficient interpretive - code, so that most simple patterns do not use much memory. However, - there is one case where the memory usage of a compiled pattern can be - unexpectedly large. If a parenthesized subpattern has a quantifier with - a minimum greater than 1 and/or a limited maximum, the whole subpattern - is repeated in the compiled code. For example, the pattern - - (abc|def){2,4} - - is compiled as if it were - - (abc|def)(abc|def)((abc|def)(abc|def)?)? - - (Technical aside: It is done this way so that backtrack points within - each of the repetitions can be independently maintained.) - - For regular expressions whose quantifiers use only small numbers, this - is not usually a problem. However, if the numbers are large, and par- - ticularly if such repetitions are nested, the memory usage can become - an embarrassment. For example, the very simple pattern - - ((ab){1,1000}c){1,3} - - uses 51K bytes when compiled using the 8-bit library. When PCRE is com- - piled with its default internal pointer size of two bytes, the size - limit on a compiled pattern is 64K data units, and this is reached with - the above pattern if the outer repetition is increased from 3 to 4. - PCRE can be compiled to use larger internal pointers and thus handle - larger compiled patterns, but it is better to try to rewrite your pat- - tern to use less memory if you can. - - One way of reducing the memory usage for such patterns is to make use - of PCRE's "subroutine" facility. Re-writing the above pattern as - - ((ab)(?2){0,999}c)(?1){0,2} - - reduces the memory requirements to 18K, and indeed it remains under 20K - even with the outer repetition increased to 100. However, this pattern - is not exactly equivalent, because the "subroutine" calls are treated - as atomic groups into which there can be no backtracking if there is a - subsequent matching failure. Therefore, PCRE cannot do this kind of - rewriting automatically. Furthermore, there is a noticeable loss of - speed when executing the modified pattern. Nevertheless, if the atomic - grouping is not a problem and the loss of speed is acceptable, this - kind of rewriting will allow you to process patterns that PCRE cannot - otherwise handle. - - -STACK USAGE AT RUN TIME - - When pcre_exec() or pcre[16|32]_exec() is used for matching, certain - kinds of pattern can cause it to use large amounts of the process - stack. In some environments the default process stack is quite small, - and if it runs out the result is often SIGSEGV. This issue is probably - the most frequently raised problem with PCRE. Rewriting your pattern - can often help. The pcrestack documentation discusses this issue in - detail. - - -PROCESSING TIME - - Certain items in regular expression patterns are processed more effi- - ciently than others. It is more efficient to use a character class like - [aeiou] than a set of single-character alternatives such as - (a|e|i|o|u). In general, the simplest construction that provides the - required behaviour is usually the most efficient. Jeffrey Friedl's book - contains a lot of useful general discussion about optimizing regular - expressions for efficient performance. This document contains a few - observations about PCRE. - - Using Unicode character properties (the \p, \P, and \X escapes) is - slow, because PCRE has to use a multi-stage table lookup whenever it - needs a character's property. If you can find an alternative pattern - that does not use character properties, it will probably be faster. - - By default, the escape sequences \b, \d, \s, and \w, and the POSIX - character classes such as [:alpha:] do not use Unicode properties, - partly for backwards compatibility, and partly for performance reasons. - However, you can set PCRE_UCP if you want Unicode character properties - to be used. This can double the matching time for items such as \d, - when matched with a traditional matching function; the performance loss - is less with a DFA matching function, and in both cases there is not - much difference for \b. - - When a pattern begins with .* not in parentheses, or in parentheses - that are not the subject of a backreference, and the PCRE_DOTALL option - is set, the pattern is implicitly anchored by PCRE, since it can match - only at the start of a subject string. However, if PCRE_DOTALL is not - set, PCRE cannot make this optimization, because the . metacharacter - does not then match a newline, and if the subject string contains new- - lines, the pattern may match from the character immediately following - one of them instead of from the very start. For example, the pattern - - .*second - - matches the subject "first\nand second" (where \n stands for a newline - character), with the match starting at the seventh character. In order - to do this, PCRE has to retry the match starting after every newline in - the subject. - - If you are using such a pattern with subject strings that do not con- - tain newlines, the best performance is obtained by setting PCRE_DOTALL, - or starting the pattern with ^.* or ^.*? to indicate explicit anchor- - ing. That saves PCRE from having to scan along the subject looking for - a newline to restart at. - - Beware of patterns that contain nested indefinite repeats. These can - take a long time to run when applied to a string that does not match. - Consider the pattern fragment - - ^(a+)* - - This can match "aaaa" in 16 different ways, and this number increases - very rapidly as the string gets longer. (The * repeat can match 0, 1, - 2, 3, or 4 times, and for each of those cases other than 0 or 4, the + - repeats can match different numbers of times.) When the remainder of - the pattern is such that the entire match is going to fail, PCRE has in - principle to try every possible variation, and this can take an - extremely long time, even for relatively short strings. - - An optimization catches some of the more simple cases such as - - (a+)*b - - where a literal character follows. Before embarking on the standard - matching procedure, PCRE checks that there is a "b" later in the sub- - ject string, and if there is not, it fails the match immediately. How- - ever, when there is no following literal this optimization cannot be - used. You can see the difference by comparing the behaviour of - - (a+)*\d - - with the pattern above. The former gives a failure almost instantly - when applied to a whole line of "a" characters, whereas the latter - takes an appreciable time with strings longer than about 20 characters. - - In many cases, the solution to this kind of performance issue is to use - an atomic group or a possessive quantifier. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 25 August 2012 - Copyright (c) 1997-2012 University of Cambridge. ------------------------------------------------------------------------------- - - -PCREPOSIX(3) Library Functions Manual PCREPOSIX(3) - - - -NAME - PCRE - Perl-compatible regular expressions. - -SYNOPSIS - - #include <pcreposix.h> - - int regcomp(regex_t *preg, const char *pattern, - int cflags); - - int regexec(regex_t *preg, const char *string, - size_t nmatch, regmatch_t pmatch[], int eflags); - size_t regerror(int errcode, const regex_t *preg, - char *errbuf, size_t errbuf_size); - - void regfree(regex_t *preg); - - -DESCRIPTION - - This set of functions provides a POSIX-style API for the PCRE regular - expression 8-bit library. See the pcreapi documentation for a descrip- - tion of PCRE's native API, which contains much additional functional- - ity. There is no POSIX-style wrapper for PCRE's 16-bit and 32-bit - library. - - The functions described here are just wrapper functions that ultimately - call the PCRE native API. Their prototypes are defined in the - pcreposix.h header file, and on Unix systems the library itself is - called pcreposix.a, so can be accessed by adding -lpcreposix to the - command for linking an application that uses them. Because the POSIX - functions call the native ones, it is also necessary to add -lpcre. - - I have implemented only those POSIX option bits that can be reasonably - mapped to PCRE native options. In addition, the option REG_EXTENDED is - defined with the value zero. This has no effect, but since programs - that are written to the POSIX interface often use it, this makes it - easier to slot in PCRE as a replacement library. Other POSIX options - are not even defined. - - There are also some other options that are not defined by POSIX. These - have been added at the request of users who want to make use of certain - PCRE-specific features via the POSIX calling interface. - - When PCRE is called via these functions, it is only the API that is - POSIX-like in style. The syntax and semantics of the regular expres- - sions themselves are still those of Perl, subject to the setting of - various PCRE options, as described below. "POSIX-like in style" means - that the API approximates to the POSIX definition; it is not fully - POSIX-compatible, and in multi-byte encoding domains it is probably - even less compatible. - - The header for these functions is supplied as pcreposix.h to avoid any - potential clash with other POSIX libraries. It can, of course, be - renamed or aliased as regex.h, which is the "correct" name. It provides - two structure types, regex_t for compiled internal forms, and reg- - match_t for returning captured substrings. It also defines some con- - stants whose names start with "REG_"; these are used for setting - options and identifying error codes. - - -COMPILING A PATTERN - - The function regcomp() is called to compile a pattern into an internal - form. The pattern is a C string terminated by a binary zero, and is - passed in the argument pattern. The preg argument is a pointer to a - regex_t structure that is used as a base for storing information about - the compiled regular expression. - - The argument cflags is either zero, or contains one or more of the bits - defined by the following macros: - - REG_DOTALL - - The PCRE_DOTALL option is set when the regular expression is passed for - compilation to the native function. Note that REG_DOTALL is not part of - the POSIX standard. - - REG_ICASE - - The PCRE_CASELESS option is set when the regular expression is passed - for compilation to the native function. - - REG_NEWLINE - - The PCRE_MULTILINE option is set when the regular expression is passed - for compilation to the native function. Note that this does not mimic - the defined POSIX behaviour for REG_NEWLINE (see the following sec- - tion). - - REG_NOSUB - - The PCRE_NO_AUTO_CAPTURE option is set when the regular expression is - passed for compilation to the native function. In addition, when a pat- - tern that is compiled with this flag is passed to regexec() for match- - ing, the nmatch and pmatch arguments are ignored, and no captured - strings are returned. - - REG_UCP - - The PCRE_UCP option is set when the regular expression is passed for - compilation to the native function. This causes PCRE to use Unicode - properties when matchine \d, \w, etc., instead of just recognizing - ASCII values. Note that REG_UTF8 is not part of the POSIX standard. - - REG_UNGREEDY - - The PCRE_UNGREEDY option is set when the regular expression is passed - for compilation to the native function. Note that REG_UNGREEDY is not - part of the POSIX standard. - - REG_UTF8 - - The PCRE_UTF8 option is set when the regular expression is passed for - compilation to the native function. This causes the pattern itself and - all data strings used for matching it to be treated as UTF-8 strings. - Note that REG_UTF8 is not part of the POSIX standard. - - In the absence of these flags, no options are passed to the native - function. This means the the regex is compiled with PCRE default - semantics. In particular, the way it handles newline characters in the - subject string is the Perl way, not the POSIX way. Note that setting - PCRE_MULTILINE has only some of the effects specified for REG_NEWLINE. - It does not affect the way newlines are matched by . (they are not) or - by a negative class such as [^a] (they are). - - The yield of regcomp() is zero on success, and non-zero otherwise. The - preg structure is filled in on success, and one member of the structure - is public: re_nsub contains the number of capturing subpatterns in the - regular expression. Various error codes are defined in the header file. - - NOTE: If the yield of regcomp() is non-zero, you must not attempt to - use the contents of the preg structure. If, for example, you pass it to - regexec(), the result is undefined and your program is likely to crash. - - -MATCHING NEWLINE CHARACTERS - - This area is not simple, because POSIX and Perl take different views of - things. It is not possible to get PCRE to obey POSIX semantics, but - then PCRE was never intended to be a POSIX engine. The following table - lists the different possibilities for matching newline characters in - PCRE: - - Default Change with - - . matches newline no PCRE_DOTALL - newline matches [^a] yes not changeable - $ matches \n at end yes PCRE_DOLLARENDONLY - $ matches \n in middle no PCRE_MULTILINE - ^ matches \n in middle no PCRE_MULTILINE - - This is the equivalent table for POSIX: - - Default Change with - - . matches newline yes REG_NEWLINE - newline matches [^a] yes REG_NEWLINE - $ matches \n at end no REG_NEWLINE - $ matches \n in middle no REG_NEWLINE - ^ matches \n in middle no REG_NEWLINE - - PCRE's behaviour is the same as Perl's, except that there is no equiva- - lent for PCRE_DOLLAR_ENDONLY in Perl. In both PCRE and Perl, there is - no way to stop newline from matching [^a]. - - The default POSIX newline handling can be obtained by setting - PCRE_DOTALL and PCRE_DOLLAR_ENDONLY, but there is no way to make PCRE - behave exactly as for the REG_NEWLINE action. - - -MATCHING A PATTERN - - The function regexec() is called to match a compiled pattern preg - against a given string, which is by default terminated by a zero byte - (but see REG_STARTEND below), subject to the options in eflags. These - can be: - - REG_NOTBOL - - The PCRE_NOTBOL option is set when calling the underlying PCRE matching - function. - - REG_NOTEMPTY - - The PCRE_NOTEMPTY option is set when calling the underlying PCRE match- - ing function. Note that REG_NOTEMPTY is not part of the POSIX standard. - However, setting this option can give more POSIX-like behaviour in some - situations. - - REG_NOTEOL - - The PCRE_NOTEOL option is set when calling the underlying PCRE matching - function. - - REG_STARTEND - - The string is considered to start at string + pmatch[0].rm_so and to - have a terminating NUL located at string + pmatch[0].rm_eo (there need - not actually be a NUL at that location), regardless of the value of - nmatch. This is a BSD extension, compatible with but not specified by - IEEE Standard 1003.2 (POSIX.2), and should be used with caution in - software intended to be portable to other systems. Note that a non-zero - rm_so does not imply REG_NOTBOL; REG_STARTEND affects only the location - of the string, not how it is matched. - - If the pattern was compiled with the REG_NOSUB flag, no data about any - matched strings is returned. The nmatch and pmatch arguments of - regexec() are ignored. - - If the value of nmatch is zero, or if the value pmatch is NULL, no data - about any matched strings is returned. - - Otherwise,the portion of the string that was matched, and also any cap- - tured substrings, are returned via the pmatch argument, which points to - an array of nmatch structures of type regmatch_t, containing the mem- - bers rm_so and rm_eo. These contain the offset to the first character - of each substring and the offset to the first character after the end - of each substring, respectively. The 0th element of the vector relates - to the entire portion of string that was matched; subsequent elements - relate to the capturing subpatterns of the regular expression. Unused - entries in the array have both structure members set to -1. - - A successful match yields a zero return; various error codes are - defined in the header file, of which REG_NOMATCH is the "expected" - failure code. - - -ERROR MESSAGES - - The regerror() function maps a non-zero errorcode from either regcomp() - or regexec() to a printable message. If preg is not NULL, the error - should have arisen from the use of that structure. A message terminated - by a binary zero is placed in errbuf. The length of the message, - including the zero, is limited to errbuf_size. The yield of the func- - tion is the size of buffer needed to hold the whole message. - - -MEMORY USAGE - - Compiling a regular expression causes memory to be allocated and asso- - ciated with the preg structure. The function regfree() frees all such - memory, after which preg may no longer be used as a compiled expres- - sion. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 09 January 2012 - Copyright (c) 1997-2012 University of Cambridge. ------------------------------------------------------------------------------- - - -PCRECPP(3) Library Functions Manual PCRECPP(3) - - - -NAME - PCRE - Perl-compatible regular expressions. - -SYNOPSIS OF C++ WRAPPER - - #include <pcrecpp.h> - - -DESCRIPTION - - The C++ wrapper for PCRE was provided by Google Inc. Some additional - functionality was added by Giuseppe Maxia. This brief man page was con- - structed from the notes in the pcrecpp.h file, which should be con- - sulted for further details. Note that the C++ wrapper supports only the - original 8-bit PCRE library. There is no 16-bit or 32-bit support at - present. - - -MATCHING INTERFACE - - The "FullMatch" operation checks that supplied text matches a supplied - pattern exactly. If pointer arguments are supplied, it copies matched - sub-strings that match sub-patterns into them. - - Example: successful match - pcrecpp::RE re("h.*o"); - re.FullMatch("hello"); - - Example: unsuccessful match (requires full match): - pcrecpp::RE re("e"); - !re.FullMatch("hello"); - - Example: creating a temporary RE object: - pcrecpp::RE("h.*o").FullMatch("hello"); - - You can pass in a "const char*" or a "string" for "text". The examples - below tend to use a const char*. You can, as in the different examples - above, store the RE object explicitly in a variable or use a temporary - RE object. The examples below use one mode or the other arbitrarily. - Either could correctly be used for any of these examples. - - You must supply extra pointer arguments to extract matched subpieces. - - Example: extracts "ruby" into "s" and 1234 into "i" - int i; - string s; - pcrecpp::RE re("(\\w+):(\\d+)"); - re.FullMatch("ruby:1234", &s, &i); - - Example: does not try to extract any extra sub-patterns - re.FullMatch("ruby:1234", &s); - - Example: does not try to extract into NULL - re.FullMatch("ruby:1234", NULL, &i); - - Example: integer overflow causes failure - !re.FullMatch("ruby:1234567891234", NULL, &i); - - Example: fails because there aren't enough sub-patterns: - !pcrecpp::RE("\\w+:\\d+").FullMatch("ruby:1234", &s); - - Example: fails because string cannot be stored in integer - !pcrecpp::RE("(.*)").FullMatch("ruby", &i); - - The provided pointer arguments can be pointers to any scalar numeric - type, or one of: - - string (matched piece is copied to string) - StringPiece (StringPiece is mutated to point to matched piece) - T (where "bool T::ParseFrom(const char*, int)" exists) - NULL (the corresponding matched sub-pattern is not copied) - - The function returns true iff all of the following conditions are sat- - isfied: - - a. "text" matches "pattern" exactly; - - b. The number of matched sub-patterns is >= number of supplied - pointers; - - c. The "i"th argument has a suitable type for holding the - string captured as the "i"th sub-pattern. If you pass in - void * NULL for the "i"th argument, or a non-void * NULL - of the correct type, or pass fewer arguments than the - number of sub-patterns, "i"th captured sub-pattern is - ignored. - - CAVEAT: An optional sub-pattern that does not exist in the matched - string is assigned the empty string. Therefore, the following will - return false (because the empty string is not a valid number): - - int number; - pcrecpp::RE::FullMatch("abc", "[a-z]+(\\d+)?", &number); - - The matching interface supports at most 16 arguments per call. If you - need more, consider using the more general interface - pcrecpp::RE::DoMatch. See pcrecpp.h for the signature for DoMatch. - - NOTE: Do not use no_arg, which is used internally to mark the end of a - list of optional arguments, as a placeholder for missing arguments, as - this can lead to segfaults. - - -QUOTING METACHARACTERS - - You can use the "QuoteMeta" operation to insert backslashes before all - potentially meaningful characters in a string. The returned string, - used as a regular expression, will exactly match the original string. - - Example: - string quoted = RE::QuoteMeta(unquoted); - - Note that it's legal to escape a character even if it has no special - meaning in a regular expression -- so this function does that. (This - also makes it identical to the perl function of the same name; see - "perldoc -f quotemeta".) For example, "1.5-2.0?" becomes - "1\.5\-2\.0\?". - - -PARTIAL MATCHES - - You can use the "PartialMatch" operation when you want the pattern to - match any substring of the text. - - Example: simple search for a string: - pcrecpp::RE("ell").PartialMatch("hello"); - - Example: find first number in a string: - int number; - pcrecpp::RE re("(\\d+)"); - re.PartialMatch("x*100 + 20", &number); - assert(number == 100); - - -UTF-8 AND THE MATCHING INTERFACE - - By default, pattern and text are plain text, one byte per character. - The UTF8 flag, passed to the constructor, causes both pattern and - string to be treated as UTF-8 text, still a byte stream but potentially - multiple bytes per character. In practice, the text is likelier to be - UTF-8 than the pattern, but the match returned may depend on the UTF8 - flag, so always use it when matching UTF8 text. For example, "." will - match one byte normally but with UTF8 set may match up to three bytes - of a multi-byte character. - - Example: - pcrecpp::RE_Options options; - options.set_utf8(); - pcrecpp::RE re(utf8_pattern, options); - re.FullMatch(utf8_string); - - Example: using the convenience function UTF8(): - pcrecpp::RE re(utf8_pattern, pcrecpp::UTF8()); - re.FullMatch(utf8_string); - - NOTE: The UTF8 flag is ignored if pcre was not configured with the - --enable-utf8 flag. - - -PASSING MODIFIERS TO THE REGULAR EXPRESSION ENGINE - - PCRE defines some modifiers to change the behavior of the regular - expression engine. The C++ wrapper defines an auxiliary class, - RE_Options, as a vehicle to pass such modifiers to a RE class. Cur- - rently, the following modifiers are supported: - - modifier description Perl corresponding - - PCRE_CASELESS case insensitive match /i - PCRE_MULTILINE multiple lines match /m - PCRE_DOTALL dot matches newlines /s - PCRE_DOLLAR_ENDONLY $ matches only at end N/A - PCRE_EXTRA strict escape parsing N/A - PCRE_EXTENDED ignore white spaces /x - PCRE_UTF8 handles UTF8 chars built-in - PCRE_UNGREEDY reverses * and *? N/A - PCRE_NO_AUTO_CAPTURE disables capturing parens N/A (*) - - (*) Both Perl and PCRE allow non capturing parentheses by means of the - "?:" modifier within the pattern itself. e.g. (?:ab|cd) does not cap- - ture, while (ab|cd) does. - - For a full account on how each modifier works, please check the PCRE - API reference page. - - For each modifier, there are two member functions whose name is made - out of the modifier in lowercase, without the "PCRE_" prefix. For - instance, PCRE_CASELESS is handled by - - bool caseless() - - which returns true if the modifier is set, and - - RE_Options & set_caseless(bool) - - which sets or unsets the modifier. Moreover, PCRE_EXTRA_MATCH_LIMIT can - be accessed through the set_match_limit() and match_limit() member - functions. Setting match_limit to a non-zero value will limit the exe- - cution of pcre to keep it from doing bad things like blowing the stack - or taking an eternity to return a result. A value of 5000 is good - enough to stop stack blowup in a 2MB thread stack. Setting match_limit - to zero disables match limiting. Alternatively, you can call - match_limit_recursion() which uses PCRE_EXTRA_MATCH_LIMIT_RECURSION to - limit how much PCRE recurses. match_limit() limits the number of - matches PCRE does; match_limit_recursion() limits the depth of internal - recursion, and therefore the amount of stack that is used. - - Normally, to pass one or more modifiers to a RE class, you declare a - RE_Options object, set the appropriate options, and pass this object to - a RE constructor. Example: - - RE_Options opt; - opt.set_caseless(true); - if (RE("HELLO", opt).PartialMatch("hello world")) ... - - RE_options has two constructors. The default constructor takes no argu- - ments and creates a set of flags that are off by default. The optional - parameter option_flags is to facilitate transfer of legacy code from C - programs. This lets you do - - RE(pattern, - RE_Options(PCRE_CASELESS|PCRE_MULTILINE)).PartialMatch(str); - - However, new code is better off doing - - RE(pattern, - RE_Options().set_caseless(true).set_multiline(true)) - .PartialMatch(str); - - If you are going to pass one of the most used modifiers, there are some - convenience functions that return a RE_Options class with the appropri- - ate modifier already set: CASELESS(), UTF8(), MULTILINE(), DOTALL(), - and EXTENDED(). - - If you need to set several options at once, and you don't want to go - through the pains of declaring a RE_Options object and setting several - options, there is a parallel method that give you such ability on the - fly. You can concatenate several set_xxxxx() member functions, since - each of them returns a reference to its class object. For example, to - pass PCRE_CASELESS, PCRE_EXTENDED, and PCRE_MULTILINE to a RE with one - statement, you may write: - - RE(" ^ xyz \\s+ .* blah$", - RE_Options() - .set_caseless(true) - .set_extended(true) - .set_multiline(true)).PartialMatch(sometext); - - -SCANNING TEXT INCREMENTALLY - - The "Consume" operation may be useful if you want to repeatedly match - regular expressions at the front of a string and skip over them as they - match. This requires use of the "StringPiece" type, which represents a - sub-range of a real string. Like RE, StringPiece is defined in the - pcrecpp namespace. - - Example: read lines of the form "var = value" from a string. - string contents = ...; // Fill string somehow - pcrecpp::StringPiece input(contents); // Wrap in a StringPiece - - string var; - int value; - pcrecpp::RE re("(\\w+) = (\\d+)\n"); - while (re.Consume(&input, &var, &value)) { - ...; - } - - Each successful call to "Consume" will set "var/value", and also - advance "input" so it points past the matched text. - - The "FindAndConsume" operation is similar to "Consume" but does not - anchor your match at the beginning of the string. For example, you - could extract all words from a string by repeatedly calling - - pcrecpp::RE("(\\w+)").FindAndConsume(&input, &word) - - -PARSING HEX/OCTAL/C-RADIX NUMBERS - - By default, if you pass a pointer to a numeric value, the corresponding - text is interpreted as a base-10 number. You can instead wrap the - pointer with a call to one of the operators Hex(), Octal(), or CRadix() - to interpret the text in another base. The CRadix operator interprets - C-style "0" (base-8) and "0x" (base-16) prefixes, but defaults to - base-10. - - Example: - int a, b, c, d; - pcrecpp::RE re("(.*) (.*) (.*) (.*)"); - re.FullMatch("100 40 0100 0x40", - pcrecpp::Octal(&a), pcrecpp::Hex(&b), - pcrecpp::CRadix(&c), pcrecpp::CRadix(&d)); - - will leave 64 in a, b, c, and d. - - -REPLACING PARTS OF STRINGS - - You can replace the first match of "pattern" in "str" with "rewrite". - Within "rewrite", backslash-escaped digits (\1 to \9) can be used to - insert text matching corresponding parenthesized group from the pat- - tern. \0 in "rewrite" refers to the entire matching text. For example: - - string s = "yabba dabba doo"; - pcrecpp::RE("b+").Replace("d", &s); - - will leave "s" containing "yada dabba doo". The result is true if the - pattern matches and a replacement occurs, false otherwise. - - GlobalReplace is like Replace except that it replaces all occurrences - of the pattern in the string with the rewrite. Replacements are not - subject to re-matching. For example: - - string s = "yabba dabba doo"; - pcrecpp::RE("b+").GlobalReplace("d", &s); - - will leave "s" containing "yada dada doo". It returns the number of - replacements made. - - Extract is like Replace, except that if the pattern matches, "rewrite" - is copied into "out" (an additional argument) with substitutions. The - non-matching portions of "text" are ignored. Returns true iff a match - occurred and the extraction happened successfully; if no match occurs, - the string is left unaffected. - - -AUTHOR - - The C++ wrapper was contributed by Google Inc. - Copyright (c) 2007 Google Inc. - - -REVISION - - Last updated: 08 January 2012 ------------------------------------------------------------------------------- - - -PCRESAMPLE(3) Library Functions Manual PCRESAMPLE(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -PCRE SAMPLE PROGRAM - - A simple, complete demonstration program, to get you started with using - PCRE, is supplied in the file pcredemo.c in the PCRE distribution. A - listing of this program is given in the pcredemo documentation. If you - do not have a copy of the PCRE distribution, you can save this listing - to re-create pcredemo.c. - - The demonstration program, which uses the original PCRE 8-bit library, - compiles the regular expression that is its first argument, and matches - it against the subject string in its second argument. No PCRE options - are set, and default character tables are used. If matching succeeds, - the program outputs the portion of the subject that matched, together - with the contents of any captured substrings. - - If the -g option is given on the command line, the program then goes on - to check for further matches of the same regular expression in the same - subject string. The logic is a little bit tricky because of the possi- - bility of matching an empty string. Comments in the code explain what - is going on. - - If PCRE is installed in the standard include and library directories - for your operating system, you should be able to compile the demonstra- - tion program using this command: - - gcc -o pcredemo pcredemo.c -lpcre - - If PCRE is installed elsewhere, you may need to add additional options - to the command line. For example, on a Unix-like system that has PCRE - installed in /usr/local, you can compile the demonstration program - using a command like this: - - gcc -o pcredemo -I/usr/local/include pcredemo.c \ - -L/usr/local/lib -lpcre - - In a Windows environment, if you want to statically link the program - against a non-dll pcre.a file, you must uncomment the line that defines - PCRE_STATIC before including pcre.h, because otherwise the pcre_mal- - loc() and pcre_free() exported functions will be declared - __declspec(dllimport), with unwanted results. - - Once you have compiled and linked the demonstration program, you can - run simple tests like this: - - ./pcredemo 'cat|dog' 'the cat sat on the mat' - ./pcredemo -g 'cat|dog' 'the dog sat on the cat' - - Note that there is a much more comprehensive test program, called - pcretest, which supports many more facilities for testing regular - expressions and both PCRE libraries. The pcredemo program is provided - as a simple coding example. - - If you try to run pcredemo when PCRE is not installed in the standard - library directory, you may get an error like this on some operating - systems (e.g. Solaris): - - ld.so.1: a.out: fatal: libpcre.so.0: open failed: No such file or - directory - - This is caused by the way shared library support works on those sys- - tems. You need to add - - -R/usr/local/lib - - (for example) to the compile command to get round this problem. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 10 January 2012 - Copyright (c) 1997-2012 University of Cambridge. ------------------------------------------------------------------------------- -PCRELIMITS(3) Library Functions Manual PCRELIMITS(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -SIZE AND OTHER LIMITATIONS - - There are some size limitations in PCRE but it is hoped that they will - never in practice be relevant. - - The maximum length of a compiled pattern is approximately 64K data - units (bytes for the 8-bit library, 16-bit units for the 16-bit - library, and 32-bit units for the 32-bit library) if PCRE is compiled - with the default internal linkage size, which is 2 bytes for the 8-bit - and 16-bit libraries, and 4 bytes for the 32-bit library. If you want - to process regular expressions that are truly enormous, you can compile - PCRE with an internal linkage size of 3 or 4 (when building the 16-bit - or 32-bit library, 3 is rounded up to 4). See the README file in the - source distribution and the pcrebuild documentation for details. In - these cases the limit is substantially larger. However, the speed of - execution is slower. - - All values in repeating quantifiers must be less than 65536. - - There is no limit to the number of parenthesized subpatterns, but there - can be no more than 65535 capturing subpatterns. There is, however, a - limit to the depth of nesting of parenthesized subpatterns of all - kinds. This is imposed in order to limit the amount of system stack - used at compile time. The limit can be specified when PCRE is built; - the default is 250. - - There is a limit to the number of forward references to subsequent sub- - patterns of around 200,000. Repeated forward references with fixed - upper limits, for example, (?2){0,100} when subpattern number 2 is to - the right, are included in the count. There is no limit to the number - of backward references. - - The maximum length of name for a named subpattern is 32 characters, and - the maximum number of named subpatterns is 10000. - - The maximum length of a name in a (*MARK), (*PRUNE), (*SKIP), or - (*THEN) verb is 255 for the 8-bit library and 65535 for the 16-bit and - 32-bit libraries. - - The maximum length of a subject string is the largest positive number - that an integer variable can hold. However, when using the traditional - matching function, PCRE uses recursion to handle subpatterns and indef- - inite repetition. This means that the available stack space may limit - the size of a subject string that can be processed by certain patterns. - For a discussion of stack issues, see the pcrestack documentation. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 05 November 2013 - Copyright (c) 1997-2013 University of Cambridge. ------------------------------------------------------------------------------- - - -PCRESTACK(3) Library Functions Manual PCRESTACK(3) - - - -NAME - PCRE - Perl-compatible regular expressions - -PCRE DISCUSSION OF STACK USAGE - - When you call pcre[16|32]_exec(), it makes use of an internal function - called match(). This calls itself recursively at branch points in the - pattern, in order to remember the state of the match so that it can - back up and try a different alternative if the first one fails. As - matching proceeds deeper and deeper into the tree of possibilities, the - recursion depth increases. The match() function is also called in other - circumstances, for example, whenever a parenthesized sub-pattern is - entered, and in certain cases of repetition. - - Not all calls of match() increase the recursion depth; for an item such - as a* it may be called several times at the same level, after matching - different numbers of a's. Furthermore, in a number of cases where the - result of the recursive call would immediately be passed back as the - result of the current call (a "tail recursion"), the function is just - restarted instead. - - The above comments apply when pcre[16|32]_exec() is run in its normal - interpretive manner. If the pattern was studied with the - PCRE_STUDY_JIT_COMPILE option, and just-in-time compiling was success- - ful, and the options passed to pcre[16|32]_exec() were not incompati- - ble, the matching process uses the JIT-compiled code instead of the - match() function. In this case, the memory requirements are handled - entirely differently. See the pcrejit documentation for details. - - The pcre[16|32]_dfa_exec() function operates in an entirely different - way, and uses recursion only when there is a regular expression recur- - sion or subroutine call in the pattern. This includes the processing of - assertion and "once-only" subpatterns, which are handled like subrou- - tine calls. Normally, these are never very deep, and the limit on the - complexity of pcre[16|32]_dfa_exec() is controlled by the amount of - workspace it is given. However, it is possible to write patterns with - runaway infinite recursions; such patterns will cause - pcre[16|32]_dfa_exec() to run out of stack. At present, there is no - protection against this. - - The comments that follow do NOT apply to pcre[16|32]_dfa_exec(); they - are relevant only for pcre[16|32]_exec() without the JIT optimization. - - Reducing pcre[16|32]_exec()'s stack usage - - Each time that match() is actually called recursively, it uses memory - from the process stack. For certain kinds of pattern and data, very - large amounts of stack may be needed, despite the recognition of "tail - recursion". You can often reduce the amount of recursion, and there- - fore the amount of stack used, by modifying the pattern that is being - matched. Consider, for example, this pattern: - - ([^<]|<(?!inet))+ - - It matches from wherever it starts until it encounters "<inet" or the - end of the data, and is the kind of pattern that might be used when - processing an XML file. Each iteration of the outer parentheses matches - either one character that is not "<" or a "<" that is not followed by - "inet". However, each time a parenthesis is processed, a recursion - occurs, so this formulation uses a stack frame for each matched charac- - ter. For a long string, a lot of stack is required. Consider now this - rewritten pattern, which matches exactly the same strings: - - ([^<]++|<(?!inet))+ - - This uses very much less stack, because runs of characters that do not - contain "<" are "swallowed" in one item inside the parentheses. Recur- - sion happens only when a "<" character that is not followed by "inet" - is encountered (and we assume this is relatively rare). A possessive - quantifier is used to stop any backtracking into the runs of non-"<" - characters, but that is not related to stack usage. - - This example shows that one way of avoiding stack problems when match- - ing long subject strings is to write repeated parenthesized subpatterns - to match more than one character whenever possible. - - Compiling PCRE to use heap instead of stack for pcre[16|32]_exec() - - In environments where stack memory is constrained, you might want to - compile PCRE to use heap memory instead of stack for remembering back- - up points when pcre[16|32]_exec() is running. This makes it run a lot - more slowly, however. Details of how to do this are given in the pcre- - build documentation. When built in this way, instead of using the - stack, PCRE obtains and frees memory by calling the functions that are - pointed to by the pcre[16|32]_stack_malloc and pcre[16|32]_stack_free - variables. By default, these point to malloc() and free(), but you can - replace the pointers to cause PCRE to use your own functions. Since the - block sizes are always the same, and are always freed in reverse order, - it may be possible to implement customized memory handlers that are - more efficient than the standard functions. - - Limiting pcre[16|32]_exec()'s stack usage - - You can set limits on the number of times that match() is called, both - in total and recursively. If a limit is exceeded, pcre[16|32]_exec() - returns an error code. Setting suitable limits should prevent it from - running out of stack. The default values of the limits are very large, - and unlikely ever to operate. They can be changed when PCRE is built, - and they can also be set when pcre[16|32]_exec() is called. For details - of these interfaces, see the pcrebuild documentation and the section on - extra data for pcre[16|32]_exec() in the pcreapi documentation. - - As a very rough rule of thumb, you should reckon on about 500 bytes per - recursion. Thus, if you want to limit your stack usage to 8Mb, you - should set the limit at 16000 recursions. A 64Mb stack, on the other - hand, can support around 128000 recursions. - - In Unix-like environments, the pcretest test program has a command line - option (-S) that can be used to increase the size of its stack. As long - as the stack is large enough, another option (-M) can be used to find - the smallest limits that allow a particular pattern to match a given - subject string. This is done by calling pcre[16|32]_exec() repeatedly - with different limits. - - Obtaining an estimate of stack usage - - The actual amount of stack used per recursion can vary quite a lot, - depending on the compiler that was used to build PCRE and the optimiza- - tion or debugging options that were set for it. The rule of thumb value - of 500 bytes mentioned above may be larger or smaller than what is - actually needed. A better approximation can be obtained by running this - command: - - pcretest -m -C - - The -C option causes pcretest to output information about the options - with which PCRE was compiled. When -m is also given (before -C), infor- - mation about stack use is given in a line like this: - - Match recursion uses stack: approximate frame size = 640 bytes - - The value is approximate because some recursions need a bit more (up to - perhaps 16 more bytes). - - If the above command is given when PCRE is compiled to use the heap - instead of the stack for recursion, the value that is output is the - size of each block that is obtained from the heap. - - Changing stack size in Unix-like systems - - In Unix-like environments, there is not often a problem with the stack - unless very long strings are involved, though the default limit on - stack size varies from system to system. Values from 8Mb to 64Mb are - common. You can find your default limit by running the command: - - ulimit -s - - Unfortunately, the effect of running out of stack is often SIGSEGV, - though sometimes a more explicit error message is given. You can nor- - mally increase the limit on stack size by code such as this: - - struct rlimit rlim; - getrlimit(RLIMIT_STACK, &rlim); - rlim.rlim_cur = 100*1024*1024; - setrlimit(RLIMIT_STACK, &rlim); - - This reads the current limits (soft and hard) using getrlimit(), then - attempts to increase the soft limit to 100Mb using setrlimit(). You - must do this before calling pcre[16|32]_exec(). - - Changing stack size in Mac OS X - - Using setrlimit(), as described above, should also work on Mac OS X. It - is also possible to set a stack size when linking a program. There is a - discussion about stack sizes in Mac OS X at this web site: - http://developer.apple.com/qa/qa2005/qa1419.html. - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 24 June 2012 - Copyright (c) 1997-2012 University of Cambridge. ------------------------------------------------------------------------------- - - diff --git a/plugins/Pcre16/docs/doc/pcre16.3 b/plugins/Pcre16/docs/doc/pcre16.3 deleted file mode 100644 index 85126a6792..0000000000 --- a/plugins/Pcre16/docs/doc/pcre16.3 +++ /dev/null @@ -1,371 +0,0 @@ -.TH PCRE 3 "12 May 2013" "PCRE 8.33" -.SH NAME -PCRE - Perl-compatible regular expressions -.sp -.B #include <pcre.h> -. -. -.SH "PCRE 16-BIT API BASIC FUNCTIONS" -.rs -.sp -.nf -.B pcre16 *pcre16_compile(PCRE_SPTR16 \fIpattern\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.sp -.B pcre16 *pcre16_compile2(PCRE_SPTR16 \fIpattern\fP, int \fIoptions\fP, -.B " int *\fIerrorcodeptr\fP," -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.sp -.B pcre16_extra *pcre16_study(const pcre16 *\fIcode\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP);" -.sp -.B void pcre16_free_study(pcre16_extra *\fIextra\fP); -.sp -.B int pcre16_exec(const pcre16 *\fIcode\fP, "const pcre16_extra *\fIextra\fP," -.B " PCRE_SPTR16 \fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP);" -.sp -.B int pcre16_dfa_exec(const pcre16 *\fIcode\fP, "const pcre16_extra *\fIextra\fP," -.B " PCRE_SPTR16 \fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP," -.B " int *\fIworkspace\fP, int \fIwscount\fP);" -.fi -. -. -.SH "PCRE 16-BIT API STRING EXTRACTION FUNCTIONS" -.rs -.sp -.nf -.B int pcre16_copy_named_substring(const pcre16 *\fIcode\fP, -.B " PCRE_SPTR16 \fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, PCRE_SPTR16 \fIstringname\fP," -.B " PCRE_UCHAR16 *\fIbuffer\fP, int \fIbuffersize\fP);" -.sp -.B int pcre16_copy_substring(PCRE_SPTR16 \fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP, PCRE_UCHAR16 *\fIbuffer\fP," -.B " int \fIbuffersize\fP);" -.sp -.B int pcre16_get_named_substring(const pcre16 *\fIcode\fP, -.B " PCRE_SPTR16 \fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, PCRE_SPTR16 \fIstringname\fP," -.B " PCRE_SPTR16 *\fIstringptr\fP);" -.sp -.B int pcre16_get_stringnumber(const pcre16 *\fIcode\fP, -.B " PCRE_SPTR16 \fIname\fP); -.sp -.B int pcre16_get_stringtable_entries(const pcre16 *\fIcode\fP, -.B " PCRE_SPTR16 \fIname\fP, PCRE_UCHAR16 **\fIfirst\fP, PCRE_UCHAR16 **\fIlast\fP);" -.sp -.B int pcre16_get_substring(PCRE_SPTR16 \fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP," -.B " PCRE_SPTR16 *\fIstringptr\fP);" -.sp -.B int pcre16_get_substring_list(PCRE_SPTR16 \fIsubject\fP, -.B " int *\fIovector\fP, int \fIstringcount\fP, PCRE_SPTR16 **\fIlistptr\fP);" -.sp -.B void pcre16_free_substring(PCRE_SPTR16 \fIstringptr\fP); -.sp -.B void pcre16_free_substring_list(PCRE_SPTR16 *\fIstringptr\fP); -.fi -. -. -.SH "PCRE 16-BIT API AUXILIARY FUNCTIONS" -.rs -.sp -.nf -.B pcre16_jit_stack *pcre16_jit_stack_alloc(int \fIstartsize\fP, int \fImaxsize\fP); -.sp -.B void pcre16_jit_stack_free(pcre16_jit_stack *\fIstack\fP); -.sp -.B void pcre16_assign_jit_stack(pcre16_extra *\fIextra\fP, -.B " pcre16_jit_callback \fIcallback\fP, void *\fIdata\fP);" -.sp -.B const unsigned char *pcre16_maketables(void); -.sp -.B int pcre16_fullinfo(const pcre16 *\fIcode\fP, "const pcre16_extra *\fIextra\fP," -.B " int \fIwhat\fP, void *\fIwhere\fP);" -.sp -.B int pcre16_refcount(pcre16 *\fIcode\fP, int \fIadjust\fP); -.sp -.B int pcre16_config(int \fIwhat\fP, void *\fIwhere\fP); -.sp -.B const char *pcre16_version(void); -.sp -.B int pcre16_pattern_to_host_byte_order(pcre16 *\fIcode\fP, -.B " pcre16_extra *\fIextra\fP, const unsigned char *\fItables\fP);" -.fi -. -. -.SH "PCRE 16-BIT API INDIRECTED FUNCTIONS" -.rs -.sp -.nf -.B void *(*pcre16_malloc)(size_t); -.sp -.B void (*pcre16_free)(void *); -.sp -.B void *(*pcre16_stack_malloc)(size_t); -.sp -.B void (*pcre16_stack_free)(void *); -.sp -.B int (*pcre16_callout)(pcre16_callout_block *); -.fi -. -. -.SH "PCRE 16-BIT API 16-BIT-ONLY FUNCTION" -.rs -.sp -.nf -.B int pcre16_utf16_to_host_byte_order(PCRE_UCHAR16 *\fIoutput\fP, -.B " PCRE_SPTR16 \fIinput\fP, int \fIlength\fP, int *\fIbyte_order\fP," -.B " int \fIkeep_boms\fP);" -.fi -. -. -.SH "THE PCRE 16-BIT LIBRARY" -.rs -.sp -Starting with release 8.30, it is possible to compile a PCRE library that -supports 16-bit character strings, including UTF-16 strings, as well as or -instead of the original 8-bit library. The majority of the work to make this -possible was done by Zoltan Herczeg. The two libraries contain identical sets -of functions, used in exactly the same way. Only the names of the functions and -the data types of their arguments and results are different. To avoid -over-complication and reduce the documentation maintenance load, most of the -PCRE documentation describes the 8-bit library, with only occasional references -to the 16-bit library. This page describes what is different when you use the -16-bit library. -.P -WARNING: A single application can be linked with both libraries, but you must -take care when processing any particular pattern to use functions from just one -library. For example, if you want to study a pattern that was compiled with -\fBpcre16_compile()\fP, you must do so with \fBpcre16_study()\fP, not -\fBpcre_study()\fP, and you must free the study data with -\fBpcre16_free_study()\fP. -. -. -.SH "THE HEADER FILE" -.rs -.sp -There is only one header file, \fBpcre.h\fP. It contains prototypes for all the -functions in all libraries, as well as definitions of flags, structures, error -codes, etc. -. -. -.SH "THE LIBRARY NAME" -.rs -.sp -In Unix-like systems, the 16-bit library is called \fBlibpcre16\fP, and can -normally be accesss by adding \fB-lpcre16\fP to the command for linking an -application that uses PCRE. -. -. -.SH "STRING TYPES" -.rs -.sp -In the 8-bit library, strings are passed to PCRE library functions as vectors -of bytes with the C type "char *". In the 16-bit library, strings are passed as -vectors of unsigned 16-bit quantities. The macro PCRE_UCHAR16 specifies an -appropriate data type, and PCRE_SPTR16 is defined as "const PCRE_UCHAR16 *". In -very many environments, "short int" is a 16-bit data type. When PCRE is built, -it defines PCRE_UCHAR16 as "unsigned short int", but checks that it really is a -16-bit data type. If it is not, the build fails with an error message telling -the maintainer to modify the definition appropriately. -. -. -.SH "STRUCTURE TYPES" -.rs -.sp -The types of the opaque structures that are used for compiled 16-bit patterns -and JIT stacks are \fBpcre16\fP and \fBpcre16_jit_stack\fP respectively. The -type of the user-accessible structure that is returned by \fBpcre16_study()\fP -is \fBpcre16_extra\fP, and the type of the structure that is used for passing -data to a callout function is \fBpcre16_callout_block\fP. These structures -contain the same fields, with the same names, as their 8-bit counterparts. The -only difference is that pointers to character strings are 16-bit instead of -8-bit types. -. -. -.SH "16-BIT FUNCTIONS" -.rs -.sp -For every function in the 8-bit library there is a corresponding function in -the 16-bit library with a name that starts with \fBpcre16_\fP instead of -\fBpcre_\fP. The prototypes are listed above. In addition, there is one extra -function, \fBpcre16_utf16_to_host_byte_order()\fP. This is a utility function -that converts a UTF-16 character string to host byte order if necessary. The -other 16-bit functions expect the strings they are passed to be in host byte -order. -.P -The \fIinput\fP and \fIoutput\fP arguments of -\fBpcre16_utf16_to_host_byte_order()\fP may point to the same address, that is, -conversion in place is supported. The output buffer must be at least as long as -the input. -.P -The \fIlength\fP argument specifies the number of 16-bit data units in the -input string; a negative value specifies a zero-terminated string. -.P -If \fIbyte_order\fP is NULL, it is assumed that the string starts off in host -byte order. This may be changed by byte-order marks (BOMs) anywhere in the -string (commonly as the first character). -.P -If \fIbyte_order\fP is not NULL, a non-zero value of the integer to which it -points means that the input starts off in host byte order, otherwise the -opposite order is assumed. Again, BOMs in the string can change this. The final -byte order is passed back at the end of processing. -.P -If \fIkeep_boms\fP is not zero, byte-order mark characters (0xfeff) are copied -into the output string. Otherwise they are discarded. -.P -The result of the function is the number of 16-bit units placed into the output -buffer, including the zero terminator if the string was zero-terminated. -. -. -.SH "SUBJECT STRING OFFSETS" -.rs -.sp -The lengths and starting offsets of subject strings must be specified in 16-bit -data units, and the offsets within subject strings that are returned by the -matching functions are in also 16-bit units rather than bytes. -. -. -.SH "NAMED SUBPATTERNS" -.rs -.sp -The name-to-number translation table that is maintained for named subpatterns -uses 16-bit characters. The \fBpcre16_get_stringtable_entries()\fP function -returns the length of each entry in the table as the number of 16-bit data -units. -. -. -.SH "OPTION NAMES" -.rs -.sp -There are two new general option names, PCRE_UTF16 and PCRE_NO_UTF16_CHECK, -which correspond to PCRE_UTF8 and PCRE_NO_UTF8_CHECK in the 8-bit library. In -fact, these new options define the same bits in the options word. There is a -discussion about the -.\" HTML <a href="pcreunicode.html#utf16strings"> -.\" </a> -validity of UTF-16 strings -.\" -in the -.\" HREF -\fBpcreunicode\fP -.\" -page. -.P -For the \fBpcre16_config()\fP function there is an option PCRE_CONFIG_UTF16 -that returns 1 if UTF-16 support is configured, otherwise 0. If this option is -given to \fBpcre_config()\fP or \fBpcre32_config()\fP, or if the -PCRE_CONFIG_UTF8 or PCRE_CONFIG_UTF32 option is given to \fBpcre16_config()\fP, -the result is the PCRE_ERROR_BADOPTION error. -. -. -.SH "CHARACTER CODES" -.rs -.sp -In 16-bit mode, when PCRE_UTF16 is not set, character values are treated in the -same way as in 8-bit, non UTF-8 mode, except, of course, that they can range -from 0 to 0xffff instead of 0 to 0xff. Character types for characters less than -0xff can therefore be influenced by the locale in the same way as before. -Characters greater than 0xff have only one case, and no "type" (such as letter -or digit). -.P -In UTF-16 mode, the character code is Unicode, in the range 0 to 0x10ffff, with -the exception of values in the range 0xd800 to 0xdfff because those are -"surrogate" values that are used in pairs to encode values greater than 0xffff. -.P -A UTF-16 string can indicate its endianness by special code knows as a -byte-order mark (BOM). The PCRE functions do not handle this, expecting strings -to be in host byte order. A utility function called -\fBpcre16_utf16_to_host_byte_order()\fP is provided to help with this (see -above). -. -. -.SH "ERROR NAMES" -.rs -.sp -The errors PCRE_ERROR_BADUTF16_OFFSET and PCRE_ERROR_SHORTUTF16 correspond to -their 8-bit counterparts. The error PCRE_ERROR_BADMODE is given when a compiled -pattern is passed to a function that processes patterns in the other -mode, for example, if a pattern compiled with \fBpcre_compile()\fP is passed to -\fBpcre16_exec()\fP. -.P -There are new error codes whose names begin with PCRE_UTF16_ERR for invalid -UTF-16 strings, corresponding to the PCRE_UTF8_ERR codes for UTF-8 strings that -are described in the section entitled -.\" HTML <a href="pcreapi.html#badutf8reasons"> -.\" </a> -"Reason codes for invalid UTF-8 strings" -.\" -in the main -.\" HREF -\fBpcreapi\fP -.\" -page. The UTF-16 errors are: -.sp - PCRE_UTF16_ERR1 Missing low surrogate at end of string - PCRE_UTF16_ERR2 Invalid low surrogate follows high surrogate - PCRE_UTF16_ERR3 Isolated low surrogate - PCRE_UTF16_ERR4 Non-character -. -. -.SH "ERROR TEXTS" -.rs -.sp -If there is an error while compiling a pattern, the error text that is passed -back by \fBpcre16_compile()\fP or \fBpcre16_compile2()\fP is still an 8-bit -character string, zero-terminated. -. -. -.SH "CALLOUTS" -.rs -.sp -The \fIsubject\fP and \fImark\fP fields in the callout block that is passed to -a callout function point to 16-bit vectors. -. -. -.SH "TESTING" -.rs -.sp -The \fBpcretest\fP program continues to operate with 8-bit input and output -files, but it can be used for testing the 16-bit library. If it is run with the -command line option \fB-16\fP, patterns and subject strings are converted from -8-bit to 16-bit before being passed to PCRE, and the 16-bit library functions -are used instead of the 8-bit ones. Returned 16-bit strings are converted to -8-bit for output. If both the 8-bit and the 32-bit libraries were not compiled, -\fBpcretest\fP defaults to 16-bit and the \fB-16\fP option is ignored. -.P -When PCRE is being built, the \fBRunTest\fP script that is called by "make -check" uses the \fBpcretest\fP \fB-C\fP option to discover which of the 8-bit, -16-bit and 32-bit libraries has been built, and runs the tests appropriately. -. -. -.SH "NOT SUPPORTED IN 16-BIT MODE" -.rs -.sp -Not all the features of the 8-bit library are available with the 16-bit -library. The C++ and POSIX wrapper functions support only the 8-bit library, -and the \fBpcregrep\fP program is at present 8-bit only. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 12 May 2013 -Copyright (c) 1997-2013 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcre32.3 b/plugins/Pcre16/docs/doc/pcre32.3 deleted file mode 100644 index 7cde8c0877..0000000000 --- a/plugins/Pcre16/docs/doc/pcre32.3 +++ /dev/null @@ -1,369 +0,0 @@ -.TH PCRE 3 "12 May 2013" "PCRE 8.33" -.SH NAME -PCRE - Perl-compatible regular expressions -.sp -.B #include <pcre.h> -. -. -.SH "PCRE 32-BIT API BASIC FUNCTIONS" -.rs -.sp -.nf -.B pcre32 *pcre32_compile(PCRE_SPTR32 \fIpattern\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.sp -.B pcre32 *pcre32_compile2(PCRE_SPTR32 \fIpattern\fP, int \fIoptions\fP, -.B " int *\fIerrorcodeptr\fP," -.B " const unsigned char *\fItableptr\fP);" -.sp -.B pcre32_extra *pcre32_study(const pcre32 *\fIcode\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP);" -.sp -.B void pcre32_free_study(pcre32_extra *\fIextra\fP); -.sp -.B int pcre32_exec(const pcre32 *\fIcode\fP, "const pcre32_extra *\fIextra\fP," -.B " PCRE_SPTR32 \fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP);" -.sp -.B int pcre32_dfa_exec(const pcre32 *\fIcode\fP, "const pcre32_extra *\fIextra\fP," -.B " PCRE_SPTR32 \fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP," -.B " int *\fIworkspace\fP, int \fIwscount\fP);" -.fi -. -. -.SH "PCRE 32-BIT API STRING EXTRACTION FUNCTIONS" -.rs -.sp -.nf -.B int pcre32_copy_named_substring(const pcre32 *\fIcode\fP, -.B " PCRE_SPTR32 \fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, PCRE_SPTR32 \fIstringname\fP," -.B " PCRE_UCHAR32 *\fIbuffer\fP, int \fIbuffersize\fP);" -.sp -.B int pcre32_copy_substring(PCRE_SPTR32 \fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP, PCRE_UCHAR32 *\fIbuffer\fP," -.B " int \fIbuffersize\fP);" -.sp -.B int pcre32_get_named_substring(const pcre32 *\fIcode\fP, -.B " PCRE_SPTR32 \fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, PCRE_SPTR32 \fIstringname\fP," -.B " PCRE_SPTR32 *\fIstringptr\fP);" -.sp -.B int pcre32_get_stringnumber(const pcre32 *\fIcode\fP, -.B " PCRE_SPTR32 \fIname\fP);" -.sp -.B int pcre32_get_stringtable_entries(const pcre32 *\fIcode\fP, -.B " PCRE_SPTR32 \fIname\fP, PCRE_UCHAR32 **\fIfirst\fP, PCRE_UCHAR32 **\fIlast\fP);" -.sp -.B int pcre32_get_substring(PCRE_SPTR32 \fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP," -.B " PCRE_SPTR32 *\fIstringptr\fP);" -.sp -.B int pcre32_get_substring_list(PCRE_SPTR32 \fIsubject\fP, -.B " int *\fIovector\fP, int \fIstringcount\fP, PCRE_SPTR32 **\fIlistptr\fP);" -.sp -.B void pcre32_free_substring(PCRE_SPTR32 \fIstringptr\fP); -.sp -.B void pcre32_free_substring_list(PCRE_SPTR32 *\fIstringptr\fP); -.fi -. -. -.SH "PCRE 32-BIT API AUXILIARY FUNCTIONS" -.rs -.sp -.nf -.B pcre32_jit_stack *pcre32_jit_stack_alloc(int \fIstartsize\fP, int \fImaxsize\fP); -.sp -.B void pcre32_jit_stack_free(pcre32_jit_stack *\fIstack\fP); -.sp -.B void pcre32_assign_jit_stack(pcre32_extra *\fIextra\fP, -.B " pcre32_jit_callback \fIcallback\fP, void *\fIdata\fP);" -.sp -.B const unsigned char *pcre32_maketables(void); -.sp -.B int pcre32_fullinfo(const pcre32 *\fIcode\fP, "const pcre32_extra *\fIextra\fP," -.B " int \fIwhat\fP, void *\fIwhere\fP);" -.sp -.B int pcre32_refcount(pcre32 *\fIcode\fP, int \fIadjust\fP); -.sp -.B int pcre32_config(int \fIwhat\fP, void *\fIwhere\fP); -.sp -.B const char *pcre32_version(void); -.sp -.B int pcre32_pattern_to_host_byte_order(pcre32 *\fIcode\fP, -.B " pcre32_extra *\fIextra\fP, const unsigned char *\fItables\fP);" -.fi -. -. -.SH "PCRE 32-BIT API INDIRECTED FUNCTIONS" -.rs -.sp -.nf -.B void *(*pcre32_malloc)(size_t); -.sp -.B void (*pcre32_free)(void *); -.sp -.B void *(*pcre32_stack_malloc)(size_t); -.sp -.B void (*pcre32_stack_free)(void *); -.sp -.B int (*pcre32_callout)(pcre32_callout_block *); -.fi -. -. -.SH "PCRE 32-BIT API 32-BIT-ONLY FUNCTION" -.rs -.sp -.nf -.B int pcre32_utf32_to_host_byte_order(PCRE_UCHAR32 *\fIoutput\fP, -.B " PCRE_SPTR32 \fIinput\fP, int \fIlength\fP, int *\fIbyte_order\fP," -.B " int \fIkeep_boms\fP);" -.fi -. -. -.SH "THE PCRE 32-BIT LIBRARY" -.rs -.sp -Starting with release 8.32, it is possible to compile a PCRE library that -supports 32-bit character strings, including UTF-32 strings, as well as or -instead of the original 8-bit library. This work was done by Christian Persch, -based on the work done by Zoltan Herczeg for the 16-bit library. All three -libraries contain identical sets of functions, used in exactly the same way. -Only the names of the functions and the data types of their arguments and -results are different. To avoid over-complication and reduce the documentation -maintenance load, most of the PCRE documentation describes the 8-bit library, -with only occasional references to the 16-bit and 32-bit libraries. This page -describes what is different when you use the 32-bit library. -.P -WARNING: A single application can be linked with all or any of the three -libraries, but you must take care when processing any particular pattern -to use functions from just one library. For example, if you want to study -a pattern that was compiled with \fBpcre32_compile()\fP, you must do so -with \fBpcre32_study()\fP, not \fBpcre_study()\fP, and you must free the -study data with \fBpcre32_free_study()\fP. -. -. -.SH "THE HEADER FILE" -.rs -.sp -There is only one header file, \fBpcre.h\fP. It contains prototypes for all the -functions in all libraries, as well as definitions of flags, structures, error -codes, etc. -. -. -.SH "THE LIBRARY NAME" -.rs -.sp -In Unix-like systems, the 32-bit library is called \fBlibpcre32\fP, and can -normally be accesss by adding \fB-lpcre32\fP to the command for linking an -application that uses PCRE. -. -. -.SH "STRING TYPES" -.rs -.sp -In the 8-bit library, strings are passed to PCRE library functions as vectors -of bytes with the C type "char *". In the 32-bit library, strings are passed as -vectors of unsigned 32-bit quantities. The macro PCRE_UCHAR32 specifies an -appropriate data type, and PCRE_SPTR32 is defined as "const PCRE_UCHAR32 *". In -very many environments, "unsigned int" is a 32-bit data type. When PCRE is -built, it defines PCRE_UCHAR32 as "unsigned int", but checks that it really is -a 32-bit data type. If it is not, the build fails with an error message telling -the maintainer to modify the definition appropriately. -. -. -.SH "STRUCTURE TYPES" -.rs -.sp -The types of the opaque structures that are used for compiled 32-bit patterns -and JIT stacks are \fBpcre32\fP and \fBpcre32_jit_stack\fP respectively. The -type of the user-accessible structure that is returned by \fBpcre32_study()\fP -is \fBpcre32_extra\fP, and the type of the structure that is used for passing -data to a callout function is \fBpcre32_callout_block\fP. These structures -contain the same fields, with the same names, as their 8-bit counterparts. The -only difference is that pointers to character strings are 32-bit instead of -8-bit types. -. -. -.SH "32-BIT FUNCTIONS" -.rs -.sp -For every function in the 8-bit library there is a corresponding function in -the 32-bit library with a name that starts with \fBpcre32_\fP instead of -\fBpcre_\fP. The prototypes are listed above. In addition, there is one extra -function, \fBpcre32_utf32_to_host_byte_order()\fP. This is a utility function -that converts a UTF-32 character string to host byte order if necessary. The -other 32-bit functions expect the strings they are passed to be in host byte -order. -.P -The \fIinput\fP and \fIoutput\fP arguments of -\fBpcre32_utf32_to_host_byte_order()\fP may point to the same address, that is, -conversion in place is supported. The output buffer must be at least as long as -the input. -.P -The \fIlength\fP argument specifies the number of 32-bit data units in the -input string; a negative value specifies a zero-terminated string. -.P -If \fIbyte_order\fP is NULL, it is assumed that the string starts off in host -byte order. This may be changed by byte-order marks (BOMs) anywhere in the -string (commonly as the first character). -.P -If \fIbyte_order\fP is not NULL, a non-zero value of the integer to which it -points means that the input starts off in host byte order, otherwise the -opposite order is assumed. Again, BOMs in the string can change this. The final -byte order is passed back at the end of processing. -.P -If \fIkeep_boms\fP is not zero, byte-order mark characters (0xfeff) are copied -into the output string. Otherwise they are discarded. -.P -The result of the function is the number of 32-bit units placed into the output -buffer, including the zero terminator if the string was zero-terminated. -. -. -.SH "SUBJECT STRING OFFSETS" -.rs -.sp -The lengths and starting offsets of subject strings must be specified in 32-bit -data units, and the offsets within subject strings that are returned by the -matching functions are in also 32-bit units rather than bytes. -. -. -.SH "NAMED SUBPATTERNS" -.rs -.sp -The name-to-number translation table that is maintained for named subpatterns -uses 32-bit characters. The \fBpcre32_get_stringtable_entries()\fP function -returns the length of each entry in the table as the number of 32-bit data -units. -. -. -.SH "OPTION NAMES" -.rs -.sp -There are two new general option names, PCRE_UTF32 and PCRE_NO_UTF32_CHECK, -which correspond to PCRE_UTF8 and PCRE_NO_UTF8_CHECK in the 8-bit library. In -fact, these new options define the same bits in the options word. There is a -discussion about the -.\" HTML <a href="pcreunicode.html#utf32strings"> -.\" </a> -validity of UTF-32 strings -.\" -in the -.\" HREF -\fBpcreunicode\fP -.\" -page. -.P -For the \fBpcre32_config()\fP function there is an option PCRE_CONFIG_UTF32 -that returns 1 if UTF-32 support is configured, otherwise 0. If this option is -given to \fBpcre_config()\fP or \fBpcre16_config()\fP, or if the -PCRE_CONFIG_UTF8 or PCRE_CONFIG_UTF16 option is given to \fBpcre32_config()\fP, -the result is the PCRE_ERROR_BADOPTION error. -. -. -.SH "CHARACTER CODES" -.rs -.sp -In 32-bit mode, when PCRE_UTF32 is not set, character values are treated in the -same way as in 8-bit, non UTF-8 mode, except, of course, that they can range -from 0 to 0x7fffffff instead of 0 to 0xff. Character types for characters less -than 0xff can therefore be influenced by the locale in the same way as before. -Characters greater than 0xff have only one case, and no "type" (such as letter -or digit). -.P -In UTF-32 mode, the character code is Unicode, in the range 0 to 0x10ffff, with -the exception of values in the range 0xd800 to 0xdfff because those are -"surrogate" values that are ill-formed in UTF-32. -.P -A UTF-32 string can indicate its endianness by special code knows as a -byte-order mark (BOM). The PCRE functions do not handle this, expecting strings -to be in host byte order. A utility function called -\fBpcre32_utf32_to_host_byte_order()\fP is provided to help with this (see -above). -. -. -.SH "ERROR NAMES" -.rs -.sp -The error PCRE_ERROR_BADUTF32 corresponds to its 8-bit counterpart. -The error PCRE_ERROR_BADMODE is given when a compiled -pattern is passed to a function that processes patterns in the other -mode, for example, if a pattern compiled with \fBpcre_compile()\fP is passed to -\fBpcre32_exec()\fP. -.P -There are new error codes whose names begin with PCRE_UTF32_ERR for invalid -UTF-32 strings, corresponding to the PCRE_UTF8_ERR codes for UTF-8 strings that -are described in the section entitled -.\" HTML <a href="pcreapi.html#badutf8reasons"> -.\" </a> -"Reason codes for invalid UTF-8 strings" -.\" -in the main -.\" HREF -\fBpcreapi\fP -.\" -page. The UTF-32 errors are: -.sp - PCRE_UTF32_ERR1 Surrogate character (range from 0xd800 to 0xdfff) - PCRE_UTF32_ERR2 Non-character - PCRE_UTF32_ERR3 Character > 0x10ffff -. -. -.SH "ERROR TEXTS" -.rs -.sp -If there is an error while compiling a pattern, the error text that is passed -back by \fBpcre32_compile()\fP or \fBpcre32_compile2()\fP is still an 8-bit -character string, zero-terminated. -. -. -.SH "CALLOUTS" -.rs -.sp -The \fIsubject\fP and \fImark\fP fields in the callout block that is passed to -a callout function point to 32-bit vectors. -. -. -.SH "TESTING" -.rs -.sp -The \fBpcretest\fP program continues to operate with 8-bit input and output -files, but it can be used for testing the 32-bit library. If it is run with the -command line option \fB-32\fP, patterns and subject strings are converted from -8-bit to 32-bit before being passed to PCRE, and the 32-bit library functions -are used instead of the 8-bit ones. Returned 32-bit strings are converted to -8-bit for output. If both the 8-bit and the 16-bit libraries were not compiled, -\fBpcretest\fP defaults to 32-bit and the \fB-32\fP option is ignored. -.P -When PCRE is being built, the \fBRunTest\fP script that is called by "make -check" uses the \fBpcretest\fP \fB-C\fP option to discover which of the 8-bit, -16-bit and 32-bit libraries has been built, and runs the tests appropriately. -. -. -.SH "NOT SUPPORTED IN 32-BIT MODE" -.rs -.sp -Not all the features of the 8-bit library are available with the 32-bit -library. The C++ and POSIX wrapper functions support only the 8-bit library, -and the \fBpcregrep\fP program is at present 8-bit only. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 12 May 2013 -Copyright (c) 1997-2013 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcre_assign_jit_stack.3 b/plugins/Pcre16/docs/doc/pcre_assign_jit_stack.3 deleted file mode 100644 index 0ecf6f2c60..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_assign_jit_stack.3 +++ /dev/null @@ -1,59 +0,0 @@ -.TH PCRE_ASSIGN_JIT_STACK 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B void pcre_assign_jit_stack(pcre_extra *\fIextra\fP, -.B " pcre_jit_callback \fIcallback\fP, void *\fIdata\fP);" -.sp -.B void pcre16_assign_jit_stack(pcre16_extra *\fIextra\fP, -.B " pcre16_jit_callback \fIcallback\fP, void *\fIdata\fP);" -.sp -.B void pcre32_assign_jit_stack(pcre32_extra *\fIextra\fP, -.B " pcre32_jit_callback \fIcallback\fP, void *\fIdata\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This function provides control over the memory used as a stack at run-time by a -call to \fBpcre[16|32]_exec()\fP with a pattern that has been successfully -compiled with JIT optimization. The arguments are: -.sp - extra the data pointer returned by \fBpcre[16|32]_study()\fP - callback a callback function - data a JIT stack or a value to be passed to the callback - function -.P -If \fIcallback\fP is NULL and \fIdata\fP is NULL, an internal 32K block on -the machine stack is used. -.P -If \fIcallback\fP is NULL and \fIdata\fP is not NULL, \fIdata\fP must -be a valid JIT stack, the result of calling \fBpcre[16|32]_jit_stack_alloc()\fP. -.P -If \fIcallback\fP not NULL, it is called with \fIdata\fP as an argument at -the start of matching, in order to set up a JIT stack. If the result is NULL, -the internal 32K stack is used; otherwise the return value must be a valid JIT -stack, the result of calling \fBpcre[16|32]_jit_stack_alloc()\fP. -.P -You may safely assign the same JIT stack to multiple patterns, as long as they -are all matched in the same thread. In a multithread application, each thread -must use its own JIT stack. For more details, see the -.\" HREF -\fBpcrejit\fP -.\" -page. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_compile.3 b/plugins/Pcre16/docs/doc/pcre_compile.3 deleted file mode 100644 index 5c16ebe26d..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_compile.3 +++ /dev/null @@ -1,96 +0,0 @@ -.TH PCRE_COMPILE 3 "01 October 2013" "PCRE 8.34" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B pcre *pcre_compile(const char *\fIpattern\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.sp -.B pcre16 *pcre16_compile(PCRE_SPTR16 \fIpattern\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.sp -.B pcre32 *pcre32_compile(PCRE_SPTR32 \fIpattern\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This function compiles a regular expression into an internal form. It is the -same as \fBpcre[16|32]_compile2()\fP, except for the absence of the -\fIerrorcodeptr\fP argument. Its arguments are: -.sp - \fIpattern\fP A zero-terminated string containing the - regular expression to be compiled - \fIoptions\fP Zero or more option bits - \fIerrptr\fP Where to put an error message - \fIerroffset\fP Offset in pattern where error was found - \fItableptr\fP Pointer to character tables, or NULL to - use the built-in default -.sp -The option bits are: -.sp - PCRE_ANCHORED Force pattern anchoring - PCRE_AUTO_CALLOUT Compile automatic callouts - PCRE_BSR_ANYCRLF \eR matches only CR, LF, or CRLF - PCRE_BSR_UNICODE \eR matches all Unicode line endings - PCRE_CASELESS Do caseless matching - PCRE_DOLLAR_ENDONLY $ not to match newline at end - PCRE_DOTALL . matches anything including NL - PCRE_DUPNAMES Allow duplicate names for subpatterns - PCRE_EXTENDED Ignore white space and # comments - PCRE_EXTRA PCRE extra features - (not much use currently) - PCRE_FIRSTLINE Force matching to be before newline - PCRE_JAVASCRIPT_COMPAT JavaScript compatibility - PCRE_MULTILINE ^ and $ match newlines within data - PCRE_NEVER_UTF Lock out UTF, e.g. via (*UTF) - PCRE_NEWLINE_ANY Recognize any Unicode newline sequence - PCRE_NEWLINE_ANYCRLF Recognize CR, LF, and CRLF as newline - sequences - PCRE_NEWLINE_CR Set CR as the newline sequence - PCRE_NEWLINE_CRLF Set CRLF as the newline sequence - PCRE_NEWLINE_LF Set LF as the newline sequence - PCRE_NO_AUTO_CAPTURE Disable numbered capturing paren- - theses (named ones available) - PCRE_NO_AUTO_POSSESS Disable auto-possessification - PCRE_NO_START_OPTIMIZE Disable match-time start optimizations - PCRE_NO_UTF16_CHECK Do not check the pattern for UTF-16 - validity (only relevant if - PCRE_UTF16 is set) - PCRE_NO_UTF32_CHECK Do not check the pattern for UTF-32 - validity (only relevant if - PCRE_UTF32 is set) - PCRE_NO_UTF8_CHECK Do not check the pattern for UTF-8 - validity (only relevant if - PCRE_UTF8 is set) - PCRE_UCP Use Unicode properties for \ed, \ew, etc. - PCRE_UNGREEDY Invert greediness of quantifiers - PCRE_UTF16 Run in \fBpcre16_compile()\fP UTF-16 mode - PCRE_UTF32 Run in \fBpcre32_compile()\fP UTF-32 mode - PCRE_UTF8 Run in \fBpcre_compile()\fP UTF-8 mode -.sp -PCRE must be built with UTF support in order to use PCRE_UTF8/16/32 and -PCRE_NO_UTF8/16/32_CHECK, and with UCP support if PCRE_UCP is used. -.P -The yield of the function is a pointer to a private data structure that -contains the compiled pattern, or NULL if an error was detected. Note that -compiling regular expressions with one version of PCRE for use with a different -version is not guaranteed to work and may cause crashes. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_compile2.3 b/plugins/Pcre16/docs/doc/pcre_compile2.3 deleted file mode 100644 index 377420180e..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_compile2.3 +++ /dev/null @@ -1,101 +0,0 @@ -.TH PCRE_COMPILE2 3 "01 October 2013" "PCRE 8.34" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B pcre *pcre_compile2(const char *\fIpattern\fP, int \fIoptions\fP, -.B " int *\fIerrorcodeptr\fP," -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.sp -.B pcre16 *pcre16_compile2(PCRE_SPTR16 \fIpattern\fP, int \fIoptions\fP, -.B " int *\fIerrorcodeptr\fP," -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.sp -.B pcre32 *pcre32_compile2(PCRE_SPTR32 \fIpattern\fP, int \fIoptions\fP, -.B " int *\fIerrorcodeptr\fP,£ -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This function compiles a regular expression into an internal form. It is the -same as \fBpcre[16|32]_compile()\fP, except for the addition of the -\fIerrorcodeptr\fP argument. The arguments are: -. -.sp - \fIpattern\fP A zero-terminated string containing the - regular expression to be compiled - \fIoptions\fP Zero or more option bits - \fIerrorcodeptr\fP Where to put an error code - \fIerrptr\fP Where to put an error message - \fIerroffset\fP Offset in pattern where error was found - \fItableptr\fP Pointer to character tables, or NULL to - use the built-in default -.sp -The option bits are: -.sp - PCRE_ANCHORED Force pattern anchoring - PCRE_AUTO_CALLOUT Compile automatic callouts - PCRE_BSR_ANYCRLF \eR matches only CR, LF, or CRLF - PCRE_BSR_UNICODE \eR matches all Unicode line endings - PCRE_CASELESS Do caseless matching - PCRE_DOLLAR_ENDONLY $ not to match newline at end - PCRE_DOTALL . matches anything including NL - PCRE_DUPNAMES Allow duplicate names for subpatterns - PCRE_EXTENDED Ignore white space and # comments - PCRE_EXTRA PCRE extra features - (not much use currently) - PCRE_FIRSTLINE Force matching to be before newline - PCRE_JAVASCRIPT_COMPAT JavaScript compatibility - PCRE_MULTILINE ^ and $ match newlines within data - PCRE_NEVER_UTF Lock out UTF, e.g. via (*UTF) - PCRE_NEWLINE_ANY Recognize any Unicode newline sequence - PCRE_NEWLINE_ANYCRLF Recognize CR, LF, and CRLF as newline - sequences - PCRE_NEWLINE_CR Set CR as the newline sequence - PCRE_NEWLINE_CRLF Set CRLF as the newline sequence - PCRE_NEWLINE_LF Set LF as the newline sequence - PCRE_NO_AUTO_CAPTURE Disable numbered capturing paren- - theses (named ones available) - PCRE_NO_AUTO_POSSESS Disable auto-possessification - PCRE_NO_START_OPTIMIZE Disable match-time start optimizations - PCRE_NO_UTF16_CHECK Do not check the pattern for UTF-16 - validity (only relevant if - PCRE_UTF16 is set) - PCRE_NO_UTF32_CHECK Do not check the pattern for UTF-32 - validity (only relevant if - PCRE_UTF32 is set) - PCRE_NO_UTF8_CHECK Do not check the pattern for UTF-8 - validity (only relevant if - PCRE_UTF8 is set) - PCRE_UCP Use Unicode properties for \ed, \ew, etc. - PCRE_UNGREEDY Invert greediness of quantifiers - PCRE_UTF16 Run \fBpcre16_compile()\fP in UTF-16 mode - PCRE_UTF32 Run \fBpcre32_compile()\fP in UTF-32 mode - PCRE_UTF8 Run \fBpcre_compile()\fP in UTF-8 mode -.sp -PCRE must be built with UTF support in order to use PCRE_UTF8/16/32 and -PCRE_NO_UTF8/16/32_CHECK, and with UCP support if PCRE_UCP is used. -.P -The yield of the function is a pointer to a private data structure that -contains the compiled pattern, or NULL if an error was detected. Note that -compiling regular expressions with one version of PCRE for use with a different -version is not guaranteed to work and may cause crashes. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_config.3 b/plugins/Pcre16/docs/doc/pcre_config.3 deleted file mode 100644 index d3de14bb73..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_config.3 +++ /dev/null @@ -1,77 +0,0 @@ -.TH PCRE_CONFIG 3 "05 November 2013" "PCRE 8.34" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.SM -.B int pcre_config(int \fIwhat\fP, void *\fIwhere\fP); -.PP -.B int pcre16_config(int \fIwhat\fP, void *\fIwhere\fP); -.PP -.B int pcre32_config(int \fIwhat\fP, void *\fIwhere\fP); -. -.SH DESCRIPTION -.rs -.sp -This function makes it possible for a client program to find out which optional -features are available in the version of the PCRE library it is using. The -arguments are as follows: -.sp - \fIwhat\fP A code specifying what information is required - \fIwhere\fP Points to where to put the data -.sp -The \fIwhere\fP argument must point to an integer variable, except for -PCRE_CONFIG_MATCH_LIMIT and PCRE_CONFIG_MATCH_LIMIT_RECURSION, when it must -point to an unsigned long integer. The available codes are: -.sp - PCRE_CONFIG_JIT Availability of just-in-time compiler - support (1=yes 0=no) - PCRE_CONFIG_JITTARGET String containing information about the - target architecture for the JIT compiler, - or NULL if there is no JIT support - PCRE_CONFIG_LINK_SIZE Internal link size: 2, 3, or 4 - PCRE_CONFIG_PARENS_LIMIT Parentheses nesting limit - PCRE_CONFIG_MATCH_LIMIT Internal resource limit - PCRE_CONFIG_MATCH_LIMIT_RECURSION - Internal recursion depth limit - PCRE_CONFIG_NEWLINE Value of the default newline sequence: - 13 (0x000d) for CR - 10 (0x000a) for LF - 3338 (0x0d0a) for CRLF - -2 for ANYCRLF - -1 for ANY - PCRE_CONFIG_BSR Indicates what \eR matches by default: - 0 all Unicode line endings - 1 CR, LF, or CRLF only - PCRE_CONFIG_POSIX_MALLOC_THRESHOLD - Threshold of return slots, above which - \fBmalloc()\fP is used by the POSIX API - PCRE_CONFIG_STACKRECURSE Recursion implementation (1=stack 0=heap) - PCRE_CONFIG_UTF16 Availability of UTF-16 support (1=yes - 0=no); option for \fBpcre16_config()\fP - PCRE_CONFIG_UTF32 Availability of UTF-32 support (1=yes - 0=no); option for \fBpcre32_config()\fP - PCRE_CONFIG_UTF8 Availability of UTF-8 support (1=yes 0=no); - option for \fBpcre_config()\fP - PCRE_CONFIG_UNICODE_PROPERTIES - Availability of Unicode property support - (1=yes 0=no) -.sp -The function yields 0 on success or PCRE_ERROR_BADOPTION otherwise. That error -is also given if PCRE_CONFIG_UTF16 or PCRE_CONFIG_UTF32 is passed to -\fBpcre_config()\fP, if PCRE_CONFIG_UTF8 or PCRE_CONFIG_UTF32 is passed to -\fBpcre16_config()\fP, or if PCRE_CONFIG_UTF8 or PCRE_CONFIG_UTF16 is passed to -\fBpcre32_config()\fP. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_copy_named_substring.3 b/plugins/Pcre16/docs/doc/pcre_copy_named_substring.3 deleted file mode 100644 index 52582aecb2..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_copy_named_substring.3 +++ /dev/null @@ -1,51 +0,0 @@ -.TH PCRE_COPY_NAMED_SUBSTRING 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre_copy_named_substring(const pcre *\fIcode\fP, -.B " const char *\fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, const char *\fIstringname\fP," -.B " char *\fIbuffer\fP, int \fIbuffersize\fP);" -.sp -.B int pcre16_copy_named_substring(const pcre16 *\fIcode\fP, -.B " PCRE_SPTR16 \fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, PCRE_SPTR16 \fIstringname\fP," -.B " PCRE_UCHAR16 *\fIbuffer\fP, int \fIbuffersize\fP);" -.sp -.B int pcre32_copy_named_substring(const pcre32 *\fIcode\fP, -.B " PCRE_SPTR32 \fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, PCRE_SPTR32 \fIstringname\fP," -.B " PCRE_UCHAR32 *\fIbuffer\fP, int \fIbuffersize\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This is a convenience function for extracting a captured substring, identified -by name, into a given buffer. The arguments are: -.sp - \fIcode\fP Pattern that was successfully matched - \fIsubject\fP Subject that has been successfully matched - \fIovector\fP Offset vector that \fBpcre[16|32]_exec()\fP used - \fIstringcount\fP Value returned by \fBpcre[16|32]_exec()\fP - \fIstringname\fP Name of the required substring - \fIbuffer\fP Buffer to receive the string - \fIbuffersize\fP Size of buffer -.sp -The yield is the length of the substring, PCRE_ERROR_NOMEMORY if the buffer was -too small, or PCRE_ERROR_NOSUBSTRING if the string name is invalid. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_copy_substring.3 b/plugins/Pcre16/docs/doc/pcre_copy_substring.3 deleted file mode 100644 index 83af6e800a..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_copy_substring.3 +++ /dev/null @@ -1,47 +0,0 @@ -.TH PCRE_COPY_SUBSTRING 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre_copy_substring(const char *\fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP, char *\fIbuffer\fP," -.B " int \fIbuffersize\fP);" -.sp -.B int pcre16_copy_substring(PCRE_SPTR16 \fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP, PCRE_UCHAR16 *\fIbuffer\fP," -.B " int \fIbuffersize\fP);" -.sp -.B int pcre32_copy_substring(PCRE_SPTR32 \fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP, PCRE_UCHAR32 *\fIbuffer\fP," -.B " int \fIbuffersize\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This is a convenience function for extracting a captured substring into a given -buffer. The arguments are: -.sp - \fIsubject\fP Subject that has been successfully matched - \fIovector\fP Offset vector that \fBpcre[16|32]_exec()\fP used - \fIstringcount\fP Value returned by \fBpcre[16|32]_exec()\fP - \fIstringnumber\fP Number of the required substring - \fIbuffer\fP Buffer to receive the string - \fIbuffersize\fP Size of buffer -.sp -The yield is the length of the string, PCRE_ERROR_NOMEMORY if the buffer was -too small, or PCRE_ERROR_NOSUBSTRING if the string number is invalid. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_dfa_exec.3 b/plugins/Pcre16/docs/doc/pcre_dfa_exec.3 deleted file mode 100644 index 39c2e836da..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_dfa_exec.3 +++ /dev/null @@ -1,118 +0,0 @@ -.TH PCRE_DFA_EXEC 3 "12 May 2013" "PCRE 8.33" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre_dfa_exec(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," -.B " const char *\fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP," -.B " int *\fIworkspace\fP, int \fIwscount\fP);" -.sp -.B int pcre16_dfa_exec(const pcre16 *\fIcode\fP, "const pcre16_extra *\fIextra\fP," -.B " PCRE_SPTR16 \fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP," -.B " int *\fIworkspace\fP, int \fIwscount\fP);" -.sp -.B int pcre32_dfa_exec(const pcre32 *\fIcode\fP, "const pcre32_extra *\fIextra\fP," -.B " PCRE_SPTR32 \fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP," -.B " int *\fIworkspace\fP, int \fIwscount\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This function matches a compiled regular expression against a given subject -string, using an alternative matching algorithm that scans the subject string -just once (\fInot\fP Perl-compatible). Note that the main, Perl-compatible, -matching function is \fBpcre[16|32]_exec()\fP. The arguments for this function -are: -.sp - \fIcode\fP Points to the compiled pattern - \fIextra\fP Points to an associated \fBpcre[16|32]_extra\fP structure, - or is NULL - \fIsubject\fP Points to the subject string - \fIlength\fP Length of the subject string - \fIstartoffset\fP Offset in the subject at which to start matching - \fIoptions\fP Option bits - \fIovector\fP Points to a vector of ints for result offsets - \fIovecsize\fP Number of elements in the vector - \fIworkspace\fP Points to a vector of ints used as working space - \fIwscount\fP Number of elements in the vector -.sp -The units for \fIlength\fP and \fIstartoffset\fP are bytes for -\fBpcre_exec()\fP, 16-bit data items for \fBpcre16_exec()\fP, and 32-bit items -for \fBpcre32_exec()\fP. The options are: -.sp - PCRE_ANCHORED Match only at the first position - PCRE_BSR_ANYCRLF \eR matches only CR, LF, or CRLF - PCRE_BSR_UNICODE \eR matches all Unicode line endings - PCRE_NEWLINE_ANY Recognize any Unicode newline sequence - PCRE_NEWLINE_ANYCRLF Recognize CR, LF, & CRLF as newline sequences - PCRE_NEWLINE_CR Recognize CR as the only newline sequence - PCRE_NEWLINE_CRLF Recognize CRLF as the only newline sequence - PCRE_NEWLINE_LF Recognize LF as the only newline sequence - PCRE_NOTBOL Subject is not the beginning of a line - PCRE_NOTEOL Subject is not the end of a line - PCRE_NOTEMPTY An empty string is not a valid match - PCRE_NOTEMPTY_ATSTART An empty string at the start of the subject - is not a valid match - PCRE_NO_START_OPTIMIZE Do not do "start-match" optimizations - PCRE_NO_UTF16_CHECK Do not check the subject for UTF-16 - validity (only relevant if PCRE_UTF16 - was set at compile time) - PCRE_NO_UTF32_CHECK Do not check the subject for UTF-32 - validity (only relevant if PCRE_UTF32 - was set at compile time) - PCRE_NO_UTF8_CHECK Do not check the subject for UTF-8 - validity (only relevant if PCRE_UTF8 - was set at compile time) - PCRE_PARTIAL ) Return PCRE_ERROR_PARTIAL for a partial - PCRE_PARTIAL_SOFT ) match if no full matches are found - PCRE_PARTIAL_HARD Return PCRE_ERROR_PARTIAL for a partial match - even if there is a full match as well - PCRE_DFA_SHORTEST Return only the shortest match - PCRE_DFA_RESTART Restart after a partial match -.sp -There are restrictions on what may appear in a pattern when using this matching -function. Details are given in the -.\" HREF -\fBpcrematching\fP -.\" -documentation. For details of partial matching, see the -.\" HREF -\fBpcrepartial\fP -.\" -page. -.P -A \fBpcre[16|32]_extra\fP structure contains the following fields: -.sp - \fIflags\fP Bits indicating which fields are set - \fIstudy_data\fP Opaque data from \fBpcre[16|32]_study()\fP - \fImatch_limit\fP Limit on internal resource use - \fImatch_limit_recursion\fP Limit on internal recursion depth - \fIcallout_data\fP Opaque data passed back to callouts - \fItables\fP Points to character tables or is NULL - \fImark\fP For passing back a *MARK pointer - \fIexecutable_jit\fP Opaque data from JIT compilation -.sp -The flag bits are PCRE_EXTRA_STUDY_DATA, PCRE_EXTRA_MATCH_LIMIT, -PCRE_EXTRA_MATCH_LIMIT_RECURSION, PCRE_EXTRA_CALLOUT_DATA, -PCRE_EXTRA_TABLES, PCRE_EXTRA_MARK and PCRE_EXTRA_EXECUTABLE_JIT. For this -matching function, the \fImatch_limit\fP and \fImatch_limit_recursion\fP fields -are not used, and must not be set. The PCRE_EXTRA_EXECUTABLE_JIT flag and -the corresponding variable are ignored. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_exec.3 b/plugins/Pcre16/docs/doc/pcre_exec.3 deleted file mode 100644 index 4686bd6de0..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_exec.3 +++ /dev/null @@ -1,99 +0,0 @@ -.TH PCRE_EXEC 3 "12 May 2013" "PCRE 8.33" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre_exec(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," -.B " const char *\fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP);" -.sp -.B int pcre16_exec(const pcre16 *\fIcode\fP, "const pcre16_extra *\fIextra\fP," -.B " PCRE_SPTR16 \fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP);" -.sp -.B int pcre32_exec(const pcre32 *\fIcode\fP, "const pcre32_extra *\fIextra\fP," -.B " PCRE_SPTR32 \fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This function matches a compiled regular expression against a given subject -string, using a matching algorithm that is similar to Perl's. It returns -offsets to captured substrings. Its arguments are: -.sp - \fIcode\fP Points to the compiled pattern - \fIextra\fP Points to an associated \fBpcre[16|32]_extra\fP structure, - or is NULL - \fIsubject\fP Points to the subject string - \fIlength\fP Length of the subject string - \fIstartoffset\fP Offset in the subject at which to start matching - \fIoptions\fP Option bits - \fIovector\fP Points to a vector of ints for result offsets - \fIovecsize\fP Number of elements in the vector (a multiple of 3) -.sp -The units for \fIlength\fP and \fIstartoffset\fP are bytes for -\fBpcre_exec()\fP, 16-bit data items for \fBpcre16_exec()\fP, and 32-bit items -for \fBpcre32_exec()\fP. The options are: -.sp - PCRE_ANCHORED Match only at the first position - PCRE_BSR_ANYCRLF \eR matches only CR, LF, or CRLF - PCRE_BSR_UNICODE \eR matches all Unicode line endings - PCRE_NEWLINE_ANY Recognize any Unicode newline sequence - PCRE_NEWLINE_ANYCRLF Recognize CR, LF, & CRLF as newline sequences - PCRE_NEWLINE_CR Recognize CR as the only newline sequence - PCRE_NEWLINE_CRLF Recognize CRLF as the only newline sequence - PCRE_NEWLINE_LF Recognize LF as the only newline sequence - PCRE_NOTBOL Subject string is not the beginning of a line - PCRE_NOTEOL Subject string is not the end of a line - PCRE_NOTEMPTY An empty string is not a valid match - PCRE_NOTEMPTY_ATSTART An empty string at the start of the subject - is not a valid match - PCRE_NO_START_OPTIMIZE Do not do "start-match" optimizations - PCRE_NO_UTF16_CHECK Do not check the subject for UTF-16 - validity (only relevant if PCRE_UTF16 - was set at compile time) - PCRE_NO_UTF32_CHECK Do not check the subject for UTF-32 - validity (only relevant if PCRE_UTF32 - was set at compile time) - PCRE_NO_UTF8_CHECK Do not check the subject for UTF-8 - validity (only relevant if PCRE_UTF8 - was set at compile time) - PCRE_PARTIAL ) Return PCRE_ERROR_PARTIAL for a partial - PCRE_PARTIAL_SOFT ) match if no full matches are found - PCRE_PARTIAL_HARD Return PCRE_ERROR_PARTIAL for a partial match - if that is found before a full match -.sp -For details of partial matching, see the -.\" HREF -\fBpcrepartial\fP -.\" -page. A \fBpcre_extra\fP structure contains the following fields: -.sp - \fIflags\fP Bits indicating which fields are set - \fIstudy_data\fP Opaque data from \fBpcre[16|32]_study()\fP - \fImatch_limit\fP Limit on internal resource use - \fImatch_limit_recursion\fP Limit on internal recursion depth - \fIcallout_data\fP Opaque data passed back to callouts - \fItables\fP Points to character tables or is NULL - \fImark\fP For passing back a *MARK pointer - \fIexecutable_jit\fP Opaque data from JIT compilation -.sp -The flag bits are PCRE_EXTRA_STUDY_DATA, PCRE_EXTRA_MATCH_LIMIT, -PCRE_EXTRA_MATCH_LIMIT_RECURSION, PCRE_EXTRA_CALLOUT_DATA, -PCRE_EXTRA_TABLES, PCRE_EXTRA_MARK and PCRE_EXTRA_EXECUTABLE_JIT. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_free_study.3 b/plugins/Pcre16/docs/doc/pcre_free_study.3 deleted file mode 100644 index 8826b73597..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_free_study.3 +++ /dev/null @@ -1,31 +0,0 @@ -.TH PCRE_FREE_STUDY 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.SM -.B void pcre_free_study(pcre_extra *\fIextra\fP); -.PP -.B void pcre16_free_study(pcre16_extra *\fIextra\fP); -.PP -.B void pcre32_free_study(pcre32_extra *\fIextra\fP); -. -.SH DESCRIPTION -.rs -.sp -This function is used to free the memory used for the data generated by a call -to \fBpcre[16|32]_study()\fP when it is no longer needed. The argument must be the -result of such a call. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_free_substring.3 b/plugins/Pcre16/docs/doc/pcre_free_substring.3 deleted file mode 100644 index 88c04019f4..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_free_substring.3 +++ /dev/null @@ -1,31 +0,0 @@ -.TH PCRE_FREE_SUBSTRING 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.SM -.B void pcre_free_substring(const char *\fIstringptr\fP); -.PP -.B void pcre16_free_substring(PCRE_SPTR16 \fIstringptr\fP); -.PP -.B void pcre32_free_substring(PCRE_SPTR32 \fIstringptr\fP); -. -.SH DESCRIPTION -.rs -.sp -This is a convenience function for freeing the store obtained by a previous -call to \fBpcre[16|32]_get_substring()\fP or \fBpcre[16|32]_get_named_substring()\fP. -Its only argument is a pointer to the string. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_free_substring_list.3 b/plugins/Pcre16/docs/doc/pcre_free_substring_list.3 deleted file mode 100644 index 248b4bd01b..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_free_substring_list.3 +++ /dev/null @@ -1,31 +0,0 @@ -.TH PCRE_FREE_SUBSTRING_LIST 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.SM -.B void pcre_free_substring_list(const char **\fIstringptr\fP); -.PP -.B void pcre16_free_substring_list(PCRE_SPTR16 *\fIstringptr\fP); -.PP -.B void pcre32_free_substring_list(PCRE_SPTR32 *\fIstringptr\fP); -. -.SH DESCRIPTION -.rs -.sp -This is a convenience function for freeing the store obtained by a previous -call to \fBpcre[16|32]_get_substring_list()\fP. Its only argument is a pointer to -the list of string pointers. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_fullinfo.3 b/plugins/Pcre16/docs/doc/pcre_fullinfo.3 deleted file mode 100644 index 01e2e92874..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_fullinfo.3 +++ /dev/null @@ -1,93 +0,0 @@ -.TH PCRE_FULLINFO 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre_fullinfo(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," -.B " int \fIwhat\fP, void *\fIwhere\fP);" -.sp -.B int pcre16_fullinfo(const pcre16 *\fIcode\fP, "const pcre16_extra *\fIextra\fP," -.B " int \fIwhat\fP, void *\fIwhere\fP);" -.sp -.B int pcre32_fullinfo(const pcre32 *\fIcode\fP, "const pcre32_extra *\fIextra\fP," -.B " int \fIwhat\fP, void *\fIwhere\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This function returns information about a compiled pattern. Its arguments are: -.sp - \fIcode\fP Compiled regular expression - \fIextra\fP Result of \fBpcre[16|32]_study()\fP or NULL - \fIwhat\fP What information is required - \fIwhere\fP Where to put the information -.sp -The following information is available: -.sp - PCRE_INFO_BACKREFMAX Number of highest back reference - PCRE_INFO_CAPTURECOUNT Number of capturing subpatterns - PCRE_INFO_DEFAULT_TABLES Pointer to default tables - PCRE_INFO_FIRSTBYTE Fixed first data unit for a match, or - -1 for start of string - or after newline, or - -2 otherwise - PCRE_INFO_FIRSTTABLE Table of first data units (after studying) - PCRE_INFO_HASCRORLF Return 1 if explicit CR or LF matches exist - PCRE_INFO_JCHANGED Return 1 if (?J) or (?-J) was used - PCRE_INFO_JIT Return 1 after successful JIT compilation - PCRE_INFO_JITSIZE Size of JIT compiled code - PCRE_INFO_LASTLITERAL Literal last data unit required - PCRE_INFO_MINLENGTH Lower bound length of matching strings - PCRE_INFO_NAMECOUNT Number of named subpatterns - PCRE_INFO_NAMEENTRYSIZE Size of name table entry - PCRE_INFO_NAMETABLE Pointer to name table - PCRE_INFO_OKPARTIAL Return 1 if partial matching can be tried - (always returns 1 after release 8.00) - PCRE_INFO_OPTIONS Option bits used for compilation - PCRE_INFO_SIZE Size of compiled pattern - PCRE_INFO_STUDYSIZE Size of study data - PCRE_INFO_FIRSTCHARACTER Fixed first data unit for a match - PCRE_INFO_FIRSTCHARACTERFLAGS Returns - 1 if there is a first data character set, which can - then be retrieved using PCRE_INFO_FIRSTCHARACTER, - 2 if the first character is at the start of the data - string or after a newline, and - 0 otherwise - PCRE_INFO_REQUIREDCHAR Literal last data unit required - PCRE_INFO_REQUIREDCHARFLAGS Returns 1 if the last data character is set (which can then - be retrieved using PCRE_INFO_REQUIREDCHAR); 0 otherwise -.sp -The \fIwhere\fP argument must point to an integer variable, except for the -following \fIwhat\fP values: -.sp - PCRE_INFO_DEFAULT_TABLES const unsigned char * - PCRE_INFO_FIRSTTABLE const unsigned char * - PCRE_INFO_NAMETABLE PCRE_SPTR16 (16-bit library) - PCRE_INFO_NAMETABLE PCRE_SPTR32 (32-bit library) - PCRE_INFO_NAMETABLE const unsigned char * (8-bit library) - PCRE_INFO_OPTIONS unsigned long int - PCRE_INFO_SIZE size_t - PCRE_INFO_FIRSTCHARACTER uint32_t - PCRE_INFO_REQUIREDCHAR uint32_t -.sp -The yield of the function is zero on success or: -.sp - PCRE_ERROR_NULL the argument \fIcode\fP was NULL - the argument \fIwhere\fP was NULL - PCRE_ERROR_BADMAGIC the "magic number" was not found - PCRE_ERROR_BADOPTION the value of \fIwhat\fP was invalid -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_get_named_substring.3 b/plugins/Pcre16/docs/doc/pcre_get_named_substring.3 deleted file mode 100644 index 84d4ee7dbb..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_get_named_substring.3 +++ /dev/null @@ -1,54 +0,0 @@ -.TH PCRE_GET_NAMED_SUBSTRING 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre_get_named_substring(const pcre *\fIcode\fP, -.B " const char *\fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, const char *\fIstringname\fP," -.B " const char **\fIstringptr\fP);" -.sp -.B int pcre16_get_named_substring(const pcre16 *\fIcode\fP, -.B " PCRE_SPTR16 \fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, PCRE_SPTR16 \fIstringname\fP," -.B " PCRE_SPTR16 *\fIstringptr\fP);" -.sp -.B int pcre32_get_named_substring(const pcre32 *\fIcode\fP, -.B " PCRE_SPTR32 \fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, PCRE_SPTR32 \fIstringname\fP," -.B " PCRE_SPTR32 *\fIstringptr\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This is a convenience function for extracting a captured substring by name. The -arguments are: -.sp - \fIcode\fP Compiled pattern - \fIsubject\fP Subject that has been successfully matched - \fIovector\fP Offset vector that \fBpcre[16|32]_exec()\fP used - \fIstringcount\fP Value returned by \fBpcre[16|32]_exec()\fP - \fIstringname\fP Name of the required substring - \fIstringptr\fP Where to put the string pointer -.sp -The memory in which the substring is placed is obtained by calling -\fBpcre[16|32]_malloc()\fP. The convenience function -\fBpcre[16|32]_free_substring()\fP can be used to free it when it is no longer -needed. The yield of the function is the length of the extracted substring, -PCRE_ERROR_NOMEMORY if sufficient memory could not be obtained, or -PCRE_ERROR_NOSUBSTRING if the string name is invalid. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_get_stringnumber.3 b/plugins/Pcre16/docs/doc/pcre_get_stringnumber.3 deleted file mode 100644 index 9fc5291dc8..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_get_stringnumber.3 +++ /dev/null @@ -1,43 +0,0 @@ -.TH PCRE_GET_STRINGNUMBER 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre_get_stringnumber(const pcre *\fIcode\fP, -.B " const char *\fIname\fP);" -.sp -.B int pcre16_get_stringnumber(const pcre16 *\fIcode\fP, -.B " PCRE_SPTR16 \fIname\fP);" -.sp -.B int pcre32_get_stringnumber(const pcre32 *\fIcode\fP, -.B " PCRE_SPTR32 \fIname\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This convenience function finds the number of a named substring capturing -parenthesis in a compiled pattern. Its arguments are: -.sp - \fIcode\fP Compiled regular expression - \fIname\fP Name whose number is required -.sp -The yield of the function is the number of the parenthesis if the name is -found, or PCRE_ERROR_NOSUBSTRING otherwise. When duplicate names are allowed -(PCRE_DUPNAMES is set), it is not defined which of the numbers is returned by -\fBpcre[16|32]_get_stringnumber()\fP. You can obtain the complete list by calling -\fBpcre[16|32]_get_stringtable_entries()\fP. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_get_stringtable_entries.3 b/plugins/Pcre16/docs/doc/pcre_get_stringtable_entries.3 deleted file mode 100644 index 5c58c90c0e..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_get_stringtable_entries.3 +++ /dev/null @@ -1,46 +0,0 @@ -.TH PCRE_GET_STRINGTABLE_ENTRIES 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre_get_stringtable_entries(const pcre *\fIcode\fP, -.B " const char *\fIname\fP, char **\fIfirst\fP, char **\fIlast\fP);" -.sp -.B int pcre16_get_stringtable_entries(const pcre16 *\fIcode\fP, -.B " PCRE_SPTR16 \fIname\fP, PCRE_UCHAR16 **\fIfirst\fP, PCRE_UCHAR16 **\fIlast\fP);" -.sp -.B int pcre32_get_stringtable_entries(const pcre32 *\fIcode\fP, -.B " PCRE_SPTR32 \fIname\fP, PCRE_UCHAR32 **\fIfirst\fP, PCRE_UCHAR32 **\fIlast\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This convenience function finds, for a compiled pattern, the first and last -entries for a given name in the table that translates capturing parenthesis -names into numbers. When names are required to be unique (PCRE_DUPNAMES is -\fInot\fP set), it is usually easier to use \fBpcre[16|32]_get_stringnumber()\fP -instead. -.sp - \fIcode\fP Compiled regular expression - \fIname\fP Name whose entries required - \fIfirst\fP Where to return a pointer to the first entry - \fIlast\fP Where to return a pointer to the last entry -.sp -The yield of the function is the length of each entry, or -PCRE_ERROR_NOSUBSTRING if none are found. -.P -There is a complete description of the PCRE native API, including the format of -the table entries, in the -.\" HREF -\fBpcreapi\fP -.\" -page, and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_get_substring.3 b/plugins/Pcre16/docs/doc/pcre_get_substring.3 deleted file mode 100644 index 1e62b2c0c6..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_get_substring.3 +++ /dev/null @@ -1,50 +0,0 @@ -.TH PCRE_GET_SUBSTRING 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre_get_substring(const char *\fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP," -.B " const char **\fIstringptr\fP);" -.sp -.B int pcre16_get_substring(PCRE_SPTR16 \fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP," -.B " PCRE_SPTR16 *\fIstringptr\fP);" -.sp -.B int pcre32_get_substring(PCRE_SPTR32 \fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP," -.B " PCRE_SPTR32 *\fIstringptr\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This is a convenience function for extracting a captured substring. The -arguments are: -.sp - \fIsubject\fP Subject that has been successfully matched - \fIovector\fP Offset vector that \fBpcre[16|32]_exec()\fP used - \fIstringcount\fP Value returned by \fBpcre[16|32]_exec()\fP - \fIstringnumber\fP Number of the required substring - \fIstringptr\fP Where to put the string pointer -.sp -The memory in which the substring is placed is obtained by calling -\fBpcre[16|32]_malloc()\fP. The convenience function -\fBpcre[16|32]_free_substring()\fP can be used to free it when it is no longer -needed. The yield of the function is the length of the substring, -PCRE_ERROR_NOMEMORY if sufficient memory could not be obtained, or -PCRE_ERROR_NOSUBSTRING if the string number is invalid. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_get_substring_list.3 b/plugins/Pcre16/docs/doc/pcre_get_substring_list.3 deleted file mode 100644 index 511a4a39d6..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_get_substring_list.3 +++ /dev/null @@ -1,47 +0,0 @@ -.TH PCRE_GET_SUBSTRING_LIST 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre_get_substring_list(const char *\fIsubject\fP, -.B " int *\fIovector\fP, int \fIstringcount\fP, const char ***\fIlistptr\fP);" -.sp -.B int pcre16_get_substring_list(PCRE_SPTR16 \fIsubject\fP, -.B " int *\fIovector\fP, int \fIstringcount\fP, PCRE_SPTR16 **\fIlistptr\fP);" -.sp -.B int pcre32_get_substring_list(PCRE_SPTR32 \fIsubject\fP, -.B " int *\fIovector\fP, int \fIstringcount\fP, PCRE_SPTR32 **\fIlistptr\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This is a convenience function for extracting a list of all the captured -substrings. The arguments are: -.sp - \fIsubject\fP Subject that has been successfully matched - \fIovector\fP Offset vector that \fBpcre[16|32]_exec\fP used - \fIstringcount\fP Value returned by \fBpcre[16|32]_exec\fP - \fIlistptr\fP Where to put a pointer to the list -.sp -The memory in which the substrings and the list are placed is obtained by -calling \fBpcre[16|32]_malloc()\fP. The convenience function -\fBpcre[16|32]_free_substring_list()\fP can be used to free it when it is no -longer needed. A pointer to a list of pointers is put in the variable whose -address is in \fIlistptr\fP. The list is terminated by a NULL pointer. The -yield of the function is zero on success or PCRE_ERROR_NOMEMORY if sufficient -memory could not be obtained. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_jit_exec.3 b/plugins/Pcre16/docs/doc/pcre_jit_exec.3 deleted file mode 100644 index ba85168178..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_jit_exec.3 +++ /dev/null @@ -1,96 +0,0 @@ -.TH PCRE_EXEC 3 "31 October 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre_jit_exec(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," -.B " const char *\fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP," -.B " pcre_jit_stack *\fIjstack\fP);" -.sp -.B int pcre16_jit_exec(const pcre16 *\fIcode\fP, "const pcre16_extra *\fIextra\fP," -.B " PCRE_SPTR16 \fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP," -.B " pcre_jit_stack *\fIjstack\fP);" -.sp -.B int pcre32_jit_exec(const pcre32 *\fIcode\fP, "const pcre32_extra *\fIextra\fP," -.B " PCRE_SPTR32 \fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP," -.B " pcre_jit_stack *\fIjstack\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This function matches a compiled regular expression that has been successfully -studied with one of the JIT options against a given subject string, using a -matching algorithm that is similar to Perl's. It is a "fast path" interface to -JIT, and it bypasses some of the sanity checks that \fBpcre_exec()\fP applies. -It returns offsets to captured substrings. Its arguments are: -.sp - \fIcode\fP Points to the compiled pattern - \fIextra\fP Points to an associated \fBpcre[16|32]_extra\fP structure, - or is NULL - \fIsubject\fP Points to the subject string - \fIlength\fP Length of the subject string, in bytes - \fIstartoffset\fP Offset in bytes in the subject at which to - start matching - \fIoptions\fP Option bits - \fIovector\fP Points to a vector of ints for result offsets - \fIovecsize\fP Number of elements in the vector (a multiple of 3) - \fIjstack\fP Pointer to a JIT stack -.sp -The allowed options are: -.sp - PCRE_NOTBOL Subject string is not the beginning of a line - PCRE_NOTEOL Subject string is not the end of a line - PCRE_NOTEMPTY An empty string is not a valid match - PCRE_NOTEMPTY_ATSTART An empty string at the start of the subject - is not a valid match - PCRE_NO_UTF16_CHECK Do not check the subject for UTF-16 - validity (only relevant if PCRE_UTF16 - was set at compile time) - PCRE_NO_UTF32_CHECK Do not check the subject for UTF-32 - validity (only relevant if PCRE_UTF32 - was set at compile time) - PCRE_NO_UTF8_CHECK Do not check the subject for UTF-8 - validity (only relevant if PCRE_UTF8 - was set at compile time) - PCRE_PARTIAL ) Return PCRE_ERROR_PARTIAL for a partial - PCRE_PARTIAL_SOFT ) match if no full matches are found - PCRE_PARTIAL_HARD Return PCRE_ERROR_PARTIAL for a partial match - if that is found before a full match -.sp -However, the PCRE_NO_UTF[8|16|32]_CHECK options have no effect, as this check -is never applied. For details of partial matching, see the -.\" HREF -\fBpcrepartial\fP -.\" -page. A \fBpcre_extra\fP structure contains the following fields: -.sp - \fIflags\fP Bits indicating which fields are set - \fIstudy_data\fP Opaque data from \fBpcre[16|32]_study()\fP - \fImatch_limit\fP Limit on internal resource use - \fImatch_limit_recursion\fP Limit on internal recursion depth - \fIcallout_data\fP Opaque data passed back to callouts - \fItables\fP Points to character tables or is NULL - \fImark\fP For passing back a *MARK pointer - \fIexecutable_jit\fP Opaque data from JIT compilation -.sp -The flag bits are PCRE_EXTRA_STUDY_DATA, PCRE_EXTRA_MATCH_LIMIT, -PCRE_EXTRA_MATCH_LIMIT_RECURSION, PCRE_EXTRA_CALLOUT_DATA, -PCRE_EXTRA_TABLES, PCRE_EXTRA_MARK and PCRE_EXTRA_EXECUTABLE_JIT. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the JIT API in the -.\" HREF -\fBpcrejit\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_jit_stack_alloc.3 b/plugins/Pcre16/docs/doc/pcre_jit_stack_alloc.3 deleted file mode 100644 index 11c97a0fc8..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_jit_stack_alloc.3 +++ /dev/null @@ -1,43 +0,0 @@ -.TH PCRE_JIT_STACK_ALLOC 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B pcre_jit_stack *pcre_jit_stack_alloc(int \fIstartsize\fP, -.B " int \fImaxsize\fP);" -.sp -.B pcre16_jit_stack *pcre16_jit_stack_alloc(int \fIstartsize\fP, -.B " int \fImaxsize\fP);" -.sp -.B pcre32_jit_stack *pcre32_jit_stack_alloc(int \fIstartsize\fP, -.B " int \fImaxsize\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This function is used to create a stack for use by the code compiled by the JIT -optimization of \fBpcre[16|32]_study()\fP. The arguments are a starting size for -the stack, and a maximum size to which it is allowed to grow. The result can be -passed to the JIT run-time code by \fBpcre[16|32]_assign_jit_stack()\fP, or that -function can set up a callback for obtaining a stack. A maximum stack size of -512K to 1M should be more than enough for any pattern. For more details, see -the -.\" HREF -\fBpcrejit\fP -.\" -page. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_jit_stack_free.3 b/plugins/Pcre16/docs/doc/pcre_jit_stack_free.3 deleted file mode 100644 index 494724e844..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_jit_stack_free.3 +++ /dev/null @@ -1,35 +0,0 @@ -.TH PCRE_JIT_STACK_FREE 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.SM -.B void pcre_jit_stack_free(pcre_jit_stack *\fIstack\fP); -.PP -.B void pcre16_jit_stack_free(pcre16_jit_stack *\fIstack\fP); -.PP -.B void pcre32_jit_stack_free(pcre32_jit_stack *\fIstack\fP); -. -.SH DESCRIPTION -.rs -.sp -This function is used to free a JIT stack that was created by -\fBpcre[16|32]_jit_stack_alloc()\fP when it is no longer needed. For more details, -see the -.\" HREF -\fBpcrejit\fP -.\" -page. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_maketables.3 b/plugins/Pcre16/docs/doc/pcre_maketables.3 deleted file mode 100644 index b2c3d23aa6..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_maketables.3 +++ /dev/null @@ -1,33 +0,0 @@ -.TH PCRE_MAKETABLES 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.SM -.B const unsigned char *pcre_maketables(void); -.PP -.B const unsigned char *pcre16_maketables(void); -.PP -.B const unsigned char *pcre32_maketables(void); -. -.SH DESCRIPTION -.rs -.sp -This function builds a set of character tables for character values less than -256. These can be passed to \fBpcre[16|32]_compile()\fP to override PCRE's -internal, built-in tables (which were made by \fBpcre[16|32]_maketables()\fP when -PCRE was compiled). You might want to do this if you are using a non-standard -locale. The function yields a pointer to the tables. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_pattern_to_host_byte_order.3 b/plugins/Pcre16/docs/doc/pcre_pattern_to_host_byte_order.3 deleted file mode 100644 index b0c41c38e8..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_pattern_to_host_byte_order.3 +++ /dev/null @@ -1,44 +0,0 @@ -.TH PCRE_PATTERN_TO_HOST_BYTE_ORDER 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre_pattern_to_host_byte_order(pcre *\fIcode\fP, -.B " pcre_extra *\fIextra\fP, const unsigned char *\fItables\fP);" -.sp -.B int pcre16_pattern_to_host_byte_order(pcre16 *\fIcode\fP, -.B " pcre16_extra *\fIextra\fP, const unsigned char *\fItables\fP);" -.sp -.B int pcre32_pattern_to_host_byte_order(pcre32 *\fIcode\fP, -.B " pcre32_extra *\fIextra\fP, const unsigned char *\fItables\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This function ensures that the bytes in 2-byte and 4-byte values in a compiled -pattern are in the correct order for the current host. It is useful when a -pattern that has been compiled on one host is transferred to another that might -have different endianness. The arguments are: -.sp - \fIcode\fP A compiled regular expression - \fIextra\fP Points to an associated \fBpcre[16|32]_extra\fP structure, - or is NULL - \fItables\fP Pointer to character tables, or NULL to - set the built-in default -.sp -The result is 0 for success, a negative PCRE_ERROR_xxx value otherwise. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_refcount.3 b/plugins/Pcre16/docs/doc/pcre_refcount.3 deleted file mode 100644 index 45a41fef6a..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_refcount.3 +++ /dev/null @@ -1,36 +0,0 @@ -.TH PCRE_REFCOUNT 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.SM -.B int pcre_refcount(pcre *\fIcode\fP, int \fIadjust\fP); -.PP -.B int pcre16_refcount(pcre16 *\fIcode\fP, int \fIadjust\fP); -.PP -.B int pcre32_refcount(pcre32 *\fIcode\fP, int \fIadjust\fP); -. -.SH DESCRIPTION -.rs -.sp -This function is used to maintain a reference count inside a data block that -contains a compiled pattern. Its arguments are: -.sp - \fIcode\fP Compiled regular expression - \fIadjust\fP Adjustment to reference value -.sp -The yield of the function is the adjusted reference value, which is constrained -to lie between 0 and 65535. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_study.3 b/plugins/Pcre16/docs/doc/pcre_study.3 deleted file mode 100644 index 1200e0a668..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_study.3 +++ /dev/null @@ -1,54 +0,0 @@ -.TH PCRE_STUDY 3 " 24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B pcre_extra *pcre_study(const pcre *\fIcode\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP);" -.sp -.B pcre16_extra *pcre16_study(const pcre16 *\fIcode\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP);" -.sp -.B pcre32_extra *pcre32_study(const pcre32 *\fIcode\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP);" -.fi -. -.SH DESCRIPTION -.rs -.sp -This function studies a compiled pattern, to see if additional information can -be extracted that might speed up matching. Its arguments are: -.sp - \fIcode\fP A compiled regular expression - \fIoptions\fP Options for \fBpcre[16|32]_study()\fP - \fIerrptr\fP Where to put an error message -.sp -If the function succeeds, it returns a value that can be passed to -\fBpcre[16|32]_exec()\fP or \fBpcre[16|32]_dfa_exec()\fP via their \fIextra\fP -arguments. -.P -If the function returns NULL, either it could not find any additional -information, or there was an error. You can tell the difference by looking at -the error value. It is NULL in first case. -.P -The only option is PCRE_STUDY_JIT_COMPILE. It requests just-in-time compilation -if possible. If PCRE has been compiled without JIT support, this option is -ignored. See the -.\" HREF -\fBpcrejit\fP -.\" -page for further details. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_utf16_to_host_byte_order.3 b/plugins/Pcre16/docs/doc/pcre_utf16_to_host_byte_order.3 deleted file mode 100644 index 1851b619da..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_utf16_to_host_byte_order.3 +++ /dev/null @@ -1,45 +0,0 @@ -.TH PCRE_UTF16_TO_HOST_BYTE_ORDER 3 "21 January 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre16_utf16_to_host_byte_order(PCRE_UCHAR16 *\fIoutput\fP, -.B " PCRE_SPTR16 \fIinput\fP, int \fIlength\fP, int *\fIhost_byte_order\fP," -.B " int \fIkeep_boms\fP);" -.fi -. -. -.SH DESCRIPTION -.rs -.sp -This function, which exists only in the 16-bit library, converts a UTF-16 -string to the correct order for the current host, taking account of any byte -order marks (BOMs) within the string. Its arguments are: -.sp - \fIoutput\fP pointer to output buffer, may be the same as \fIinput\fP - \fIinput\fP pointer to input buffer - \fIlength\fP number of 16-bit units in the input, or negative for - a zero-terminated string - \fIhost_byte_order\fP a NULL value or a non-zero value pointed to means - start in host byte order - \fIkeep_boms\fP if non-zero, BOMs are copied to the output string -.sp -The result of the function is the number of 16-bit units placed into the output -buffer, including the zero terminator if the string was zero-terminated. -.P -If \fIhost_byte_order\fP is not NULL, it is set to indicate the byte order that -is current at the end of the string. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_utf32_to_host_byte_order.3 b/plugins/Pcre16/docs/doc/pcre_utf32_to_host_byte_order.3 deleted file mode 100644 index a415dcf5fa..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_utf32_to_host_byte_order.3 +++ /dev/null @@ -1,45 +0,0 @@ -.TH PCRE_UTF32_TO_HOST_BYTE_ORDER 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.nf -.B int pcre32_utf32_to_host_byte_order(PCRE_UCHAR32 *\fIoutput\fP, -.B " PCRE_SPTR32 \fIinput\fP, int \fIlength\fP, int *\fIhost_byte_order\fP," -.B " int \fIkeep_boms\fP);" -.fi -. -. -.SH DESCRIPTION -.rs -.sp -This function, which exists only in the 32-bit library, converts a UTF-32 -string to the correct order for the current host, taking account of any byte -order marks (BOMs) within the string. Its arguments are: -.sp - \fIoutput\fP pointer to output buffer, may be the same as \fIinput\fP - \fIinput\fP pointer to input buffer - \fIlength\fP number of 32-bit units in the input, or negative for - a zero-terminated string - \fIhost_byte_order\fP a NULL value or a non-zero value pointed to means - start in host byte order - \fIkeep_boms\fP if non-zero, BOMs are copied to the output string -.sp -The result of the function is the number of 32-bit units placed into the output -buffer, including the zero terminator if the string was zero-terminated. -.P -If \fIhost_byte_order\fP is not NULL, it is set to indicate the byte order that -is current at the end of the string. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcre_version.3 b/plugins/Pcre16/docs/doc/pcre_version.3 deleted file mode 100644 index 0f4973f9c7..0000000000 --- a/plugins/Pcre16/docs/doc/pcre_version.3 +++ /dev/null @@ -1,31 +0,0 @@ -.TH PCRE_VERSION 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.SM -.B const char *pcre_version(void); -.PP -.B const char *pcre16_version(void); -.PP -.B const char *pcre32_version(void); -. -.SH DESCRIPTION -.rs -.sp -This function (even in the 16-bit and 32-bit libraries) returns a -zero-terminated, 8-bit character string that gives the version number of the -PCRE library and the date of its release. -.P -There is a complete description of the PCRE native API in the -.\" HREF -\fBpcreapi\fP -.\" -page and a description of the POSIX API in the -.\" HREF -\fBpcreposix\fP -.\" -page. diff --git a/plugins/Pcre16/docs/doc/pcreapi.3 b/plugins/Pcre16/docs/doc/pcreapi.3 deleted file mode 100644 index ab3eaa0b52..0000000000 --- a/plugins/Pcre16/docs/doc/pcreapi.3 +++ /dev/null @@ -1,2919 +0,0 @@ -.TH PCREAPI 3 "09 February 2014" "PCRE 8.35" -.SH NAME -PCRE - Perl-compatible regular expressions -.sp -.B #include <pcre.h> -. -. -.SH "PCRE NATIVE API BASIC FUNCTIONS" -.rs -.sp -.nf -.B pcre *pcre_compile(const char *\fIpattern\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.sp -.B pcre *pcre_compile2(const char *\fIpattern\fP, int \fIoptions\fP, -.B " int *\fIerrorcodeptr\fP," -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.sp -.B pcre_extra *pcre_study(const pcre *\fIcode\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP);" -.sp -.B void pcre_free_study(pcre_extra *\fIextra\fP); -.sp -.B int pcre_exec(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," -.B " const char *\fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP);" -.sp -.B int pcre_dfa_exec(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," -.B " const char *\fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP," -.B " int *\fIworkspace\fP, int \fIwscount\fP);" -.fi -. -. -.SH "PCRE NATIVE API STRING EXTRACTION FUNCTIONS" -.rs -.sp -.nf -.B int pcre_copy_named_substring(const pcre *\fIcode\fP, -.B " const char *\fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, const char *\fIstringname\fP," -.B " char *\fIbuffer\fP, int \fIbuffersize\fP);" -.sp -.B int pcre_copy_substring(const char *\fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP, char *\fIbuffer\fP," -.B " int \fIbuffersize\fP);" -.sp -.B int pcre_get_named_substring(const pcre *\fIcode\fP, -.B " const char *\fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, const char *\fIstringname\fP," -.B " const char **\fIstringptr\fP);" -.sp -.B int pcre_get_stringnumber(const pcre *\fIcode\fP, -.B " const char *\fIname\fP);" -.sp -.B int pcre_get_stringtable_entries(const pcre *\fIcode\fP, -.B " const char *\fIname\fP, char **\fIfirst\fP, char **\fIlast\fP);" -.sp -.B int pcre_get_substring(const char *\fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP," -.B " const char **\fIstringptr\fP);" -.sp -.B int pcre_get_substring_list(const char *\fIsubject\fP, -.B " int *\fIovector\fP, int \fIstringcount\fP, const char ***\fIlistptr\fP);" -.sp -.B void pcre_free_substring(const char *\fIstringptr\fP); -.sp -.B void pcre_free_substring_list(const char **\fIstringptr\fP); -.fi -. -. -.SH "PCRE NATIVE API AUXILIARY FUNCTIONS" -.rs -.sp -.nf -.B int pcre_jit_exec(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," -.B " const char *\fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP," -.B " pcre_jit_stack *\fIjstack\fP);" -.sp -.B pcre_jit_stack *pcre_jit_stack_alloc(int \fIstartsize\fP, int \fImaxsize\fP); -.sp -.B void pcre_jit_stack_free(pcre_jit_stack *\fIstack\fP); -.sp -.B void pcre_assign_jit_stack(pcre_extra *\fIextra\fP, -.B " pcre_jit_callback \fIcallback\fP, void *\fIdata\fP);" -.sp -.B const unsigned char *pcre_maketables(void); -.sp -.B int pcre_fullinfo(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," -.B " int \fIwhat\fP, void *\fIwhere\fP);" -.sp -.B int pcre_refcount(pcre *\fIcode\fP, int \fIadjust\fP); -.sp -.B int pcre_config(int \fIwhat\fP, void *\fIwhere\fP); -.sp -.B const char *pcre_version(void); -.sp -.B int pcre_pattern_to_host_byte_order(pcre *\fIcode\fP, -.B " pcre_extra *\fIextra\fP, const unsigned char *\fItables\fP);" -.fi -. -. -.SH "PCRE NATIVE API INDIRECTED FUNCTIONS" -.rs -.sp -.nf -.B void *(*pcre_malloc)(size_t); -.sp -.B void (*pcre_free)(void *); -.sp -.B void *(*pcre_stack_malloc)(size_t); -.sp -.B void (*pcre_stack_free)(void *); -.sp -.B int (*pcre_callout)(pcre_callout_block *); -.sp -.B int (*pcre_stack_guard)(void); -.fi -. -. -.SH "PCRE 8-BIT, 16-BIT, AND 32-BIT LIBRARIES" -.rs -.sp -As well as support for 8-bit character strings, PCRE also supports 16-bit -strings (from release 8.30) and 32-bit strings (from release 8.32), by means of -two additional libraries. They can be built as well as, or instead of, the -8-bit library. To avoid too much complication, this document describes the -8-bit versions of the functions, with only occasional references to the 16-bit -and 32-bit libraries. -.P -The 16-bit and 32-bit functions operate in the same way as their 8-bit -counterparts; they just use different data types for their arguments and -results, and their names start with \fBpcre16_\fP or \fBpcre32_\fP instead of -\fBpcre_\fP. For every option that has UTF8 in its name (for example, -PCRE_UTF8), there are corresponding 16-bit and 32-bit names with UTF8 replaced -by UTF16 or UTF32, respectively. This facility is in fact just cosmetic; the -16-bit and 32-bit option names define the same bit values. -.P -References to bytes and UTF-8 in this document should be read as references to -16-bit data units and UTF-16 when using the 16-bit library, or 32-bit data -units and UTF-32 when using the 32-bit library, unless specified otherwise. -More details of the specific differences for the 16-bit and 32-bit libraries -are given in the -.\" HREF -\fBpcre16\fP -.\" -and -.\" HREF -\fBpcre32\fP -.\" -pages. -. -. -.SH "PCRE API OVERVIEW" -.rs -.sp -PCRE has its own native API, which is described in this document. There are -also some wrapper functions (for the 8-bit library only) that correspond to the -POSIX regular expression API, but they do not give access to all the -functionality. They are described in the -.\" HREF -\fBpcreposix\fP -.\" -documentation. Both of these APIs define a set of C function calls. A C++ -wrapper (again for the 8-bit library only) is also distributed with PCRE. It is -documented in the -.\" HREF -\fBpcrecpp\fP -.\" -page. -.P -The native API C function prototypes are defined in the header file -\fBpcre.h\fP, and on Unix-like systems the (8-bit) library itself is called -\fBlibpcre\fP. It can normally be accessed by adding \fB-lpcre\fP to the -command for linking an application that uses PCRE. The header file defines the -macros PCRE_MAJOR and PCRE_MINOR to contain the major and minor release numbers -for the library. Applications can use these to include support for different -releases of PCRE. -.P -In a Windows environment, if you want to statically link an application program -against a non-dll \fBpcre.a\fP file, you must define PCRE_STATIC before -including \fBpcre.h\fP or \fBpcrecpp.h\fP, because otherwise the -\fBpcre_malloc()\fP and \fBpcre_free()\fP exported functions will be declared -\fB__declspec(dllimport)\fP, with unwanted results. -.P -The functions \fBpcre_compile()\fP, \fBpcre_compile2()\fP, \fBpcre_study()\fP, -and \fBpcre_exec()\fP are used for compiling and matching regular expressions -in a Perl-compatible manner. A sample program that demonstrates the simplest -way of using them is provided in the file called \fIpcredemo.c\fP in the PCRE -source distribution. A listing of this program is given in the -.\" HREF -\fBpcredemo\fP -.\" -documentation, and the -.\" HREF -\fBpcresample\fP -.\" -documentation describes how to compile and run it. -.P -Just-in-time compiler support is an optional feature of PCRE that can be built -in appropriate hardware environments. It greatly speeds up the matching -performance of many patterns. Simple programs can easily request that it be -used if available, by setting an option that is ignored when it is not -relevant. More complicated programs might need to make use of the functions -\fBpcre_jit_stack_alloc()\fP, \fBpcre_jit_stack_free()\fP, and -\fBpcre_assign_jit_stack()\fP in order to control the JIT code's memory usage. -.P -From release 8.32 there is also a direct interface for JIT execution, which -gives improved performance. The JIT-specific functions are discussed in the -.\" HREF -\fBpcrejit\fP -.\" -documentation. -.P -A second matching function, \fBpcre_dfa_exec()\fP, which is not -Perl-compatible, is also provided. This uses a different algorithm for the -matching. The alternative algorithm finds all possible matches (at a given -point in the subject), and scans the subject just once (unless there are -lookbehind assertions). However, this algorithm does not return captured -substrings. A description of the two matching algorithms and their advantages -and disadvantages is given in the -.\" HREF -\fBpcrematching\fP -.\" -documentation. -.P -In addition to the main compiling and matching functions, there are convenience -functions for extracting captured substrings from a subject string that is -matched by \fBpcre_exec()\fP. They are: -.sp - \fBpcre_copy_substring()\fP - \fBpcre_copy_named_substring()\fP - \fBpcre_get_substring()\fP - \fBpcre_get_named_substring()\fP - \fBpcre_get_substring_list()\fP - \fBpcre_get_stringnumber()\fP - \fBpcre_get_stringtable_entries()\fP -.sp -\fBpcre_free_substring()\fP and \fBpcre_free_substring_list()\fP are also -provided, to free the memory used for extracted strings. -.P -The function \fBpcre_maketables()\fP is used to build a set of character tables -in the current locale for passing to \fBpcre_compile()\fP, \fBpcre_exec()\fP, -or \fBpcre_dfa_exec()\fP. This is an optional facility that is provided for -specialist use. Most commonly, no special tables are passed, in which case -internal tables that are generated when PCRE is built are used. -.P -The function \fBpcre_fullinfo()\fP is used to find out information about a -compiled pattern. The function \fBpcre_version()\fP returns a pointer to a -string containing the version of PCRE and its date of release. -.P -The function \fBpcre_refcount()\fP maintains a reference count in a data block -containing a compiled pattern. This is provided for the benefit of -object-oriented applications. -.P -The global variables \fBpcre_malloc\fP and \fBpcre_free\fP initially contain -the entry points of the standard \fBmalloc()\fP and \fBfree()\fP functions, -respectively. PCRE calls the memory management functions via these variables, -so a calling program can replace them if it wishes to intercept the calls. This -should be done before calling any PCRE functions. -.P -The global variables \fBpcre_stack_malloc\fP and \fBpcre_stack_free\fP are also -indirections to memory management functions. These special functions are used -only when PCRE is compiled to use the heap for remembering data, instead of -recursive function calls, when running the \fBpcre_exec()\fP function. See the -.\" HREF -\fBpcrebuild\fP -.\" -documentation for details of how to do this. It is a non-standard way of -building PCRE, for use in environments that have limited stacks. Because of the -greater use of memory management, it runs more slowly. Separate functions are -provided so that special-purpose external code can be used for this case. When -used, these functions are always called in a stack-like manner (last obtained, -first freed), and always for memory blocks of the same size. There is a -discussion about PCRE's stack usage in the -.\" HREF -\fBpcrestack\fP -.\" -documentation. -.P -The global variable \fBpcre_callout\fP initially contains NULL. It can be set -by the caller to a "callout" function, which PCRE will then call at specified -points during a matching operation. Details are given in the -.\" HREF -\fBpcrecallout\fP -.\" -documentation. -.P -The global variable \fBpcre_stack_guard\fP initially contains NULL. It can be -set by the caller to a function that is called by PCRE whenever it starts -to compile a parenthesized part of a pattern. When parentheses are nested, PCRE -uses recursive function calls, which use up the system stack. This function is -provided so that applications with restricted stacks can force a compilation -error if the stack runs out. The function should return zero if all is well, or -non-zero to force an error. -. -. -.\" HTML <a name="newlines"></a> -.SH NEWLINES -.rs -.sp -PCRE supports five different conventions for indicating line breaks in -strings: a single CR (carriage return) character, a single LF (linefeed) -character, the two-character sequence CRLF, any of the three preceding, or any -Unicode newline sequence. The Unicode newline sequences are the three just -mentioned, plus the single characters VT (vertical tab, U+000B), FF (form feed, -U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS -(paragraph separator, U+2029). -.P -Each of the first three conventions is used by at least one operating system as -its standard newline sequence. When PCRE is built, a default can be specified. -The default default is LF, which is the Unix standard. When PCRE is run, the -default can be overridden, either when a pattern is compiled, or when it is -matched. -.P -At compile time, the newline convention can be specified by the \fIoptions\fP -argument of \fBpcre_compile()\fP, or it can be specified by special text at the -start of the pattern itself; this overrides any other settings. See the -.\" HREF -\fBpcrepattern\fP -.\" -page for details of the special character sequences. -.P -In the PCRE documentation the word "newline" is used to mean "the character or -pair of characters that indicate a line break". The choice of newline -convention affects the handling of the dot, circumflex, and dollar -metacharacters, the handling of #-comments in /x mode, and, when CRLF is a -recognized line ending sequence, the match position advancement for a -non-anchored pattern. There is more detail about this in the -.\" HTML <a href="#execoptions"> -.\" </a> -section on \fBpcre_exec()\fP options -.\" -below. -.P -The choice of newline convention does not affect the interpretation of -the \en or \er escape sequences, nor does it affect what \eR matches, which is -controlled in a similar way, but by separate options. -. -. -.SH MULTITHREADING -.rs -.sp -The PCRE functions can be used in multi-threading applications, with the -proviso that the memory management functions pointed to by \fBpcre_malloc\fP, -\fBpcre_free\fP, \fBpcre_stack_malloc\fP, and \fBpcre_stack_free\fP, and the -callout and stack-checking functions pointed to by \fBpcre_callout\fP and -\fBpcre_stack_guard\fP, are shared by all threads. -.P -The compiled form of a regular expression is not altered during matching, so -the same compiled pattern can safely be used by several threads at once. -.P -If the just-in-time optimization feature is being used, it needs separate -memory stack areas for each thread. See the -.\" HREF -\fBpcrejit\fP -.\" -documentation for more details. -. -. -.SH "SAVING PRECOMPILED PATTERNS FOR LATER USE" -.rs -.sp -The compiled form of a regular expression can be saved and re-used at a later -time, possibly by a different program, and even on a host other than the one on -which it was compiled. Details are given in the -.\" HREF -\fBpcreprecompile\fP -.\" -documentation, which includes a description of the -\fBpcre_pattern_to_host_byte_order()\fP function. However, compiling a regular -expression with one version of PCRE for use with a different version is not -guaranteed to work and may cause crashes. -. -. -.SH "CHECKING BUILD-TIME OPTIONS" -.rs -.sp -.B int pcre_config(int \fIwhat\fP, void *\fIwhere\fP); -.PP -The function \fBpcre_config()\fP makes it possible for a PCRE client to -discover which optional features have been compiled into the PCRE library. The -.\" HREF -\fBpcrebuild\fP -.\" -documentation has more details about these optional features. -.P -The first argument for \fBpcre_config()\fP is an integer, specifying which -information is required; the second argument is a pointer to a variable into -which the information is placed. The returned value is zero on success, or the -negative error code PCRE_ERROR_BADOPTION if the value in the first argument is -not recognized. The following information is available: -.sp - PCRE_CONFIG_UTF8 -.sp -The output is an integer that is set to one if UTF-8 support is available; -otherwise it is set to zero. This value should normally be given to the 8-bit -version of this function, \fBpcre_config()\fP. If it is given to the 16-bit -or 32-bit version of this function, the result is PCRE_ERROR_BADOPTION. -.sp - PCRE_CONFIG_UTF16 -.sp -The output is an integer that is set to one if UTF-16 support is available; -otherwise it is set to zero. This value should normally be given to the 16-bit -version of this function, \fBpcre16_config()\fP. If it is given to the 8-bit -or 32-bit version of this function, the result is PCRE_ERROR_BADOPTION. -.sp - PCRE_CONFIG_UTF32 -.sp -The output is an integer that is set to one if UTF-32 support is available; -otherwise it is set to zero. This value should normally be given to the 32-bit -version of this function, \fBpcre32_config()\fP. If it is given to the 8-bit -or 16-bit version of this function, the result is PCRE_ERROR_BADOPTION. -.sp - PCRE_CONFIG_UNICODE_PROPERTIES -.sp -The output is an integer that is set to one if support for Unicode character -properties is available; otherwise it is set to zero. -.sp - PCRE_CONFIG_JIT -.sp -The output is an integer that is set to one if support for just-in-time -compiling is available; otherwise it is set to zero. -.sp - PCRE_CONFIG_JITTARGET -.sp -The output is a pointer to a zero-terminated "const char *" string. If JIT -support is available, the string contains the name of the architecture for -which the JIT compiler is configured, for example "x86 32bit (little endian + -unaligned)". If JIT support is not available, the result is NULL. -.sp - PCRE_CONFIG_NEWLINE -.sp -The output is an integer whose value specifies the default character sequence -that is recognized as meaning "newline". The values that are supported in -ASCII/Unicode environments are: 10 for LF, 13 for CR, 3338 for CRLF, -2 for -ANYCRLF, and -1 for ANY. In EBCDIC environments, CR, ANYCRLF, and ANY yield the -same values. However, the value for LF is normally 21, though some EBCDIC -environments use 37. The corresponding values for CRLF are 3349 and 3365. The -default should normally correspond to the standard sequence for your operating -system. -.sp - PCRE_CONFIG_BSR -.sp -The output is an integer whose value indicates what character sequences the \eR -escape sequence matches by default. A value of 0 means that \eR matches any -Unicode line ending sequence; a value of 1 means that \eR matches only CR, LF, -or CRLF. The default can be overridden when a pattern is compiled or matched. -.sp - PCRE_CONFIG_LINK_SIZE -.sp -The output is an integer that contains the number of bytes used for internal -linkage in compiled regular expressions. For the 8-bit library, the value can -be 2, 3, or 4. For the 16-bit library, the value is either 2 or 4 and is still -a number of bytes. For the 32-bit library, the value is either 2 or 4 and is -still a number of bytes. The default value of 2 is sufficient for all but the -most massive patterns, since it allows the compiled pattern to be up to 64K in -size. Larger values allow larger regular expressions to be compiled, at the -expense of slower matching. -.sp - PCRE_CONFIG_POSIX_MALLOC_THRESHOLD -.sp -The output is an integer that contains the threshold above which the POSIX -interface uses \fBmalloc()\fP for output vectors. Further details are given in -the -.\" HREF -\fBpcreposix\fP -.\" -documentation. -.sp - PCRE_CONFIG_PARENS_LIMIT -.sp -The output is a long integer that gives the maximum depth of nesting of -parentheses (of any kind) in a pattern. This limit is imposed to cap the amount -of system stack used when a pattern is compiled. It is specified when PCRE is -built; the default is 250. This limit does not take into account the stack that -may already be used by the calling application. For finer control over -compilation stack usage, you can set a pointer to an external checking function -in \fBpcre_stack_guard\fP. -.sp - PCRE_CONFIG_MATCH_LIMIT -.sp -The output is a long integer that gives the default limit for the number of -internal matching function calls in a \fBpcre_exec()\fP execution. Further -details are given with \fBpcre_exec()\fP below. -.sp - PCRE_CONFIG_MATCH_LIMIT_RECURSION -.sp -The output is a long integer that gives the default limit for the depth of -recursion when calling the internal matching function in a \fBpcre_exec()\fP -execution. Further details are given with \fBpcre_exec()\fP below. -.sp - PCRE_CONFIG_STACKRECURSE -.sp -The output is an integer that is set to one if internal recursion when running -\fBpcre_exec()\fP is implemented by recursive function calls that use the stack -to remember their state. This is the usual way that PCRE is compiled. The -output is zero if PCRE was compiled to use blocks of data on the heap instead -of recursive function calls. In this case, \fBpcre_stack_malloc\fP and -\fBpcre_stack_free\fP are called to manage memory blocks on the heap, thus -avoiding the use of the stack. -. -. -.SH "COMPILING A PATTERN" -.rs -.sp -.nf -.B pcre *pcre_compile(const char *\fIpattern\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.sp -.B pcre *pcre_compile2(const char *\fIpattern\fP, int \fIoptions\fP, -.B " int *\fIerrorcodeptr\fP," -.B " const char **\fIerrptr\fP, int *\fIerroffset\fP," -.B " const unsigned char *\fItableptr\fP);" -.fi -.P -Either of the functions \fBpcre_compile()\fP or \fBpcre_compile2()\fP can be -called to compile a pattern into an internal form. The only difference between -the two interfaces is that \fBpcre_compile2()\fP has an additional argument, -\fIerrorcodeptr\fP, via which a numerical error code can be returned. To avoid -too much repetition, we refer just to \fBpcre_compile()\fP below, but the -information applies equally to \fBpcre_compile2()\fP. -.P -The pattern is a C string terminated by a binary zero, and is passed in the -\fIpattern\fP argument. A pointer to a single block of memory that is obtained -via \fBpcre_malloc\fP is returned. This contains the compiled code and related -data. The \fBpcre\fP type is defined for the returned block; this is a typedef -for a structure whose contents are not externally defined. It is up to the -caller to free the memory (via \fBpcre_free\fP) when it is no longer required. -.P -Although the compiled code of a PCRE regex is relocatable, that is, it does not -depend on memory location, the complete \fBpcre\fP data block is not -fully relocatable, because it may contain a copy of the \fItableptr\fP -argument, which is an address (see below). -.P -The \fIoptions\fP argument contains various bit settings that affect the -compilation. It should be zero if no options are required. The available -options are described below. Some of them (in particular, those that are -compatible with Perl, but some others as well) can also be set and unset from -within the pattern (see the detailed description in the -.\" HREF -\fBpcrepattern\fP -.\" -documentation). For those options that can be different in different parts of -the pattern, the contents of the \fIoptions\fP argument specifies their -settings at the start of compilation and execution. The PCRE_ANCHORED, -PCRE_BSR_\fIxxx\fP, PCRE_NEWLINE_\fIxxx\fP, PCRE_NO_UTF8_CHECK, and -PCRE_NO_START_OPTIMIZE options can be set at the time of matching as well as at -compile time. -.P -If \fIerrptr\fP is NULL, \fBpcre_compile()\fP returns NULL immediately. -Otherwise, if compilation of a pattern fails, \fBpcre_compile()\fP returns -NULL, and sets the variable pointed to by \fIerrptr\fP to point to a textual -error message. This is a static string that is part of the library. You must -not try to free it. Normally, the offset from the start of the pattern to the -data unit that was being processed when the error was discovered is placed in -the variable pointed to by \fIerroffset\fP, which must not be NULL (if it is, -an immediate error is given). However, for an invalid UTF-8 or UTF-16 string, -the offset is that of the first data unit of the failing character. -.P -Some errors are not detected until the whole pattern has been scanned; in these -cases, the offset passed back is the length of the pattern. Note that the -offset is in data units, not characters, even in a UTF mode. It may sometimes -point into the middle of a UTF-8 or UTF-16 character. -.P -If \fBpcre_compile2()\fP is used instead of \fBpcre_compile()\fP, and the -\fIerrorcodeptr\fP argument is not NULL, a non-zero error code number is -returned via this argument in the event of an error. This is in addition to the -textual error message. Error codes and messages are listed below. -.P -If the final argument, \fItableptr\fP, is NULL, PCRE uses a default set of -character tables that are built when PCRE is compiled, using the default C -locale. Otherwise, \fItableptr\fP must be an address that is the result of a -call to \fBpcre_maketables()\fP. This value is stored with the compiled -pattern, and used again by \fBpcre_exec()\fP and \fBpcre_dfa_exec()\fP when the -pattern is matched. For more discussion, see the section on locale support -below. -.P -This code fragment shows a typical straightforward call to \fBpcre_compile()\fP: -.sp - pcre *re; - const char *error; - int erroffset; - re = pcre_compile( - "^A.*Z", /* the pattern */ - 0, /* default options */ - &error, /* for error message */ - &erroffset, /* for error offset */ - NULL); /* use default character tables */ -.sp -The following names for option bits are defined in the \fBpcre.h\fP header -file: -.sp - PCRE_ANCHORED -.sp -If this bit is set, the pattern is forced to be "anchored", that is, it is -constrained to match only at the first matching point in the string that is -being searched (the "subject string"). This effect can also be achieved by -appropriate constructs in the pattern itself, which is the only way to do it in -Perl. -.sp - PCRE_AUTO_CALLOUT -.sp -If this bit is set, \fBpcre_compile()\fP automatically inserts callout items, -all with number 255, before each pattern item. For discussion of the callout -facility, see the -.\" HREF -\fBpcrecallout\fP -.\" -documentation. -.sp - PCRE_BSR_ANYCRLF - PCRE_BSR_UNICODE -.sp -These options (which are mutually exclusive) control what the \eR escape -sequence matches. The choice is either to match only CR, LF, or CRLF, or to -match any Unicode newline sequence. The default is specified when PCRE is -built. It can be overridden from within the pattern, or by setting an option -when a compiled pattern is matched. -.sp - PCRE_CASELESS -.sp -If this bit is set, letters in the pattern match both upper and lower case -letters. It is equivalent to Perl's /i option, and it can be changed within a -pattern by a (?i) option setting. In UTF-8 mode, PCRE always understands the -concept of case for characters whose values are less than 128, so caseless -matching is always possible. For characters with higher values, the concept of -case is supported if PCRE is compiled with Unicode property support, but not -otherwise. If you want to use caseless matching for characters 128 and above, -you must ensure that PCRE is compiled with Unicode property support as well as -with UTF-8 support. -.sp - PCRE_DOLLAR_ENDONLY -.sp -If this bit is set, a dollar metacharacter in the pattern matches only at the -end of the subject string. Without this option, a dollar also matches -immediately before a newline at the end of the string (but not before any other -newlines). The PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is set. -There is no equivalent to this option in Perl, and no way to set it within a -pattern. -.sp - PCRE_DOTALL -.sp -If this bit is set, a dot metacharacter in the pattern matches a character of -any value, including one that indicates a newline. However, it only ever -matches one character, even if newlines are coded as CRLF. Without this option, -a dot does not match when the current position is at a newline. This option is -equivalent to Perl's /s option, and it can be changed within a pattern by a -(?s) option setting. A negative class such as [^a] always matches newline -characters, independent of the setting of this option. -.sp - PCRE_DUPNAMES -.sp -If this bit is set, names used to identify capturing subpatterns need not be -unique. This can be helpful for certain types of pattern when it is known that -only one instance of the named subpattern can ever be matched. There are more -details of named subpatterns below; see also the -.\" HREF -\fBpcrepattern\fP -.\" -documentation. -.sp - PCRE_EXTENDED -.sp -If this bit is set, most white space characters in the pattern are totally -ignored except when escaped or inside a character class. However, white space -is not allowed within sequences such as (?> that introduce various -parenthesized subpatterns, nor within a numerical quantifier such as {1,3}. -However, ignorable white space is permitted between an item and a following -quantifier and between a quantifier and a following + that indicates -possessiveness. -.P -White space did not used to include the VT character (code 11), because Perl -did not treat this character as white space. However, Perl changed at release -5.18, so PCRE followed at release 8.34, and VT is now treated as white space. -.P -PCRE_EXTENDED also causes characters between an unescaped # outside a character -class and the next newline, inclusive, to be ignored. PCRE_EXTENDED is -equivalent to Perl's /x option, and it can be changed within a pattern by a -(?x) option setting. -.P -Which characters are interpreted as newlines is controlled by the options -passed to \fBpcre_compile()\fP or by a special sequence at the start of the -pattern, as described in the section entitled -.\" HTML <a href="pcrepattern.html#newlines"> -.\" </a> -"Newline conventions" -.\" -in the \fBpcrepattern\fP documentation. Note that the end of this type of -comment is a literal newline sequence in the pattern; escape sequences that -happen to represent a newline do not count. -.P -This option makes it possible to include comments inside complicated patterns. -Note, however, that this applies only to data characters. White space characters -may never appear within special character sequences in a pattern, for example -within the sequence (?( that introduces a conditional subpattern. -.sp - PCRE_EXTRA -.sp -This option was invented in order to turn on additional functionality of PCRE -that is incompatible with Perl, but it is currently of very little use. When -set, any backslash in a pattern that is followed by a letter that has no -special meaning causes an error, thus reserving these combinations for future -expansion. By default, as in Perl, a backslash followed by a letter with no -special meaning is treated as a literal. (Perl can, however, be persuaded to -give an error for this, by running it with the -w option.) There are at present -no other features controlled by this option. It can also be set by a (?X) -option setting within a pattern. -.sp - PCRE_FIRSTLINE -.sp -If this option is set, an unanchored pattern is required to match before or at -the first newline in the subject string, though the matched text may continue -over the newline. -.sp - PCRE_JAVASCRIPT_COMPAT -.sp -If this option is set, PCRE's behaviour is changed in some ways so that it is -compatible with JavaScript rather than Perl. The changes are as follows: -.P -(1) A lone closing square bracket in a pattern causes a compile-time error, -because this is illegal in JavaScript (by default it is treated as a data -character). Thus, the pattern AB]CD becomes illegal when this option is set. -.P -(2) At run time, a back reference to an unset subpattern group matches an empty -string (by default this causes the current matching alternative to fail). A -pattern such as (\e1)(a) succeeds when this option is set (assuming it can find -an "a" in the subject), whereas it fails by default, for Perl compatibility. -.P -(3) \eU matches an upper case "U" character; by default \eU causes a compile -time error (Perl uses \eU to upper case subsequent characters). -.P -(4) \eu matches a lower case "u" character unless it is followed by four -hexadecimal digits, in which case the hexadecimal number defines the code point -to match. By default, \eu causes a compile time error (Perl uses it to upper -case the following character). -.P -(5) \ex matches a lower case "x" character unless it is followed by two -hexadecimal digits, in which case the hexadecimal number defines the code point -to match. By default, as in Perl, a hexadecimal number is always expected after -\ex, but it may have zero, one, or two digits (so, for example, \exz matches a -binary zero character followed by z). -.sp - PCRE_MULTILINE -.sp -By default, for the purposes of matching "start of line" and "end of line", -PCRE treats the subject string as consisting of a single line of characters, -even if it actually contains newlines. The "start of line" metacharacter (^) -matches only at the start of the string, and the "end of line" metacharacter -($) matches only at the end of the string, or before a terminating newline -(except when PCRE_DOLLAR_ENDONLY is set). Note, however, that unless -PCRE_DOTALL is set, the "any character" metacharacter (.) does not match at a -newline. This behaviour (for ^, $, and dot) is the same as Perl. -.P -When PCRE_MULTILINE it is set, the "start of line" and "end of line" constructs -match immediately following or immediately before internal newlines in the -subject string, respectively, as well as at the very start and end. This is -equivalent to Perl's /m option, and it can be changed within a pattern by a -(?m) option setting. If there are no newlines in a subject string, or no -occurrences of ^ or $ in a pattern, setting PCRE_MULTILINE has no effect. -.sp - PCRE_NEVER_UTF -.sp -This option locks out interpretation of the pattern as UTF-8 (or UTF-16 or -UTF-32 in the 16-bit and 32-bit libraries). In particular, it prevents the -creator of the pattern from switching to UTF interpretation by starting the -pattern with (*UTF). This may be useful in applications that process patterns -from external sources. The combination of PCRE_UTF8 and PCRE_NEVER_UTF also -causes an error. -.sp - PCRE_NEWLINE_CR - PCRE_NEWLINE_LF - PCRE_NEWLINE_CRLF - PCRE_NEWLINE_ANYCRLF - PCRE_NEWLINE_ANY -.sp -These options override the default newline definition that was chosen when PCRE -was built. Setting the first or the second specifies that a newline is -indicated by a single character (CR or LF, respectively). Setting -PCRE_NEWLINE_CRLF specifies that a newline is indicated by the two-character -CRLF sequence. Setting PCRE_NEWLINE_ANYCRLF specifies that any of the three -preceding sequences should be recognized. Setting PCRE_NEWLINE_ANY specifies -that any Unicode newline sequence should be recognized. -.P -In an ASCII/Unicode environment, the Unicode newline sequences are the three -just mentioned, plus the single characters VT (vertical tab, U+000B), FF (form -feed, U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS -(paragraph separator, U+2029). For the 8-bit library, the last two are -recognized only in UTF-8 mode. -.P -When PCRE is compiled to run in an EBCDIC (mainframe) environment, the code for -CR is 0x0d, the same as ASCII. However, the character code for LF is normally -0x15, though in some EBCDIC environments 0x25 is used. Whichever of these is -not LF is made to correspond to Unicode's NEL character. EBCDIC codes are all -less than 256. For more details, see the -.\" HREF -\fBpcrebuild\fP -.\" -documentation. -.P -The newline setting in the options word uses three bits that are treated -as a number, giving eight possibilities. Currently only six are used (default -plus the five values above). This means that if you set more than one newline -option, the combination may or may not be sensible. For example, -PCRE_NEWLINE_CR with PCRE_NEWLINE_LF is equivalent to PCRE_NEWLINE_CRLF, but -other combinations may yield unused numbers and cause an error. -.P -The only time that a line break in a pattern is specially recognized when -compiling is when PCRE_EXTENDED is set. CR and LF are white space characters, -and so are ignored in this mode. Also, an unescaped # outside a character class -indicates a comment that lasts until after the next line break sequence. In -other circumstances, line break sequences in patterns are treated as literal -data. -.P -The newline option that is set at compile time becomes the default that is used -for \fBpcre_exec()\fP and \fBpcre_dfa_exec()\fP, but it can be overridden. -.sp - PCRE_NO_AUTO_CAPTURE -.sp -If this option is set, it disables the use of numbered capturing parentheses in -the pattern. Any opening parenthesis that is not followed by ? behaves as if it -were followed by ?: but named parentheses can still be used for capturing (and -they acquire numbers in the usual way). There is no equivalent of this option -in Perl. -.sp - PCRE_NO_AUTO_POSSESS -.sp -If this option is set, it disables "auto-possessification". This is an -optimization that, for example, turns a+b into a++b in order to avoid -backtracks into a+ that can never be successful. However, if callouts are in -use, auto-possessification means that some of them are never taken. You can set -this option if you want the matching functions to do a full unoptimized search -and run all the callouts, but it is mainly provided for testing purposes. -.sp - PCRE_NO_START_OPTIMIZE -.sp -This is an option that acts at matching time; that is, it is really an option -for \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP. If it is set at compile time, -it is remembered with the compiled pattern and assumed at matching time. This -is necessary if you want to use JIT execution, because the JIT compiler needs -to know whether or not this option is set. For details see the discussion of -PCRE_NO_START_OPTIMIZE -.\" HTML <a href="#execoptions"> -.\" </a> -below. -.\" -.sp - PCRE_UCP -.sp -This option changes the way PCRE processes \eB, \eb, \eD, \ed, \eS, \es, \eW, -\ew, and some of the POSIX character classes. By default, only ASCII characters -are recognized, but if PCRE_UCP is set, Unicode properties are used instead to -classify characters. More details are given in the section on -.\" HTML <a href="pcre.html#genericchartypes"> -.\" </a> -generic character types -.\" -in the -.\" HREF -\fBpcrepattern\fP -.\" -page. If you set PCRE_UCP, matching one of the items it affects takes much -longer. The option is available only if PCRE has been compiled with Unicode -property support. -.sp - PCRE_UNGREEDY -.sp -This option inverts the "greediness" of the quantifiers so that they are not -greedy by default, but become greedy if followed by "?". It is not compatible -with Perl. It can also be set by a (?U) option setting within the pattern. -.sp - PCRE_UTF8 -.sp -This option causes PCRE to regard both the pattern and the subject as strings -of UTF-8 characters instead of single-byte strings. However, it is available -only when PCRE is built to include UTF support. If not, the use of this option -provokes an error. Details of how this option changes the behaviour of PCRE are -given in the -.\" HREF -\fBpcreunicode\fP -.\" -page. -.sp - PCRE_NO_UTF8_CHECK -.sp -When PCRE_UTF8 is set, the validity of the pattern as a UTF-8 string is -automatically checked. There is a discussion about the -.\" HTML <a href="pcreunicode.html#utf8strings"> -.\" </a> -validity of UTF-8 strings -.\" -in the -.\" HREF -\fBpcreunicode\fP -.\" -page. If an invalid UTF-8 sequence is found, \fBpcre_compile()\fP returns an -error. If you already know that your pattern is valid, and you want to skip -this check for performance reasons, you can set the PCRE_NO_UTF8_CHECK option. -When it is set, the effect of passing an invalid UTF-8 string as a pattern is -undefined. It may cause your program to crash or loop. Note that this option -can also be passed to \fBpcre_exec()\fP and \fBpcre_dfa_exec()\fP, to suppress -the validity checking of subject strings only. If the same string is being -matched many times, the option can be safely set for the second and subsequent -matchings to improve performance. -. -. -.SH "COMPILATION ERROR CODES" -.rs -.sp -The following table lists the error codes than may be returned by -\fBpcre_compile2()\fP, along with the error messages that may be returned by -both compiling functions. Note that error messages are always 8-bit ASCII -strings, even in 16-bit or 32-bit mode. As PCRE has developed, some error codes -have fallen out of use. To avoid confusion, they have not been re-used. -.sp - 0 no error - 1 \e at end of pattern - 2 \ec at end of pattern - 3 unrecognized character follows \e - 4 numbers out of order in {} quantifier - 5 number too big in {} quantifier - 6 missing terminating ] for character class - 7 invalid escape sequence in character class - 8 range out of order in character class - 9 nothing to repeat - 10 [this code is not in use] - 11 internal error: unexpected repeat - 12 unrecognized character after (? or (?- - 13 POSIX named classes are supported only within a class - 14 missing ) - 15 reference to non-existent subpattern - 16 erroffset passed as NULL - 17 unknown option bit(s) set - 18 missing ) after comment - 19 [this code is not in use] - 20 regular expression is too large - 21 failed to get memory - 22 unmatched parentheses - 23 internal error: code overflow - 24 unrecognized character after (?< - 25 lookbehind assertion is not fixed length - 26 malformed number or name after (?( - 27 conditional group contains more than two branches - 28 assertion expected after (?( - 29 (?R or (?[+-]digits must be followed by ) - 30 unknown POSIX class name - 31 POSIX collating elements are not supported - 32 this version of PCRE is compiled without UTF support - 33 [this code is not in use] - 34 character value in \ex{} or \eo{} is too large - 35 invalid condition (?(0) - 36 \eC not allowed in lookbehind assertion - 37 PCRE does not support \eL, \el, \eN{name}, \eU, or \eu - 38 number after (?C is > 255 - 39 closing ) for (?C expected - 40 recursive call could loop indefinitely - 41 unrecognized character after (?P - 42 syntax error in subpattern name (missing terminator) - 43 two named subpatterns have the same name - 44 invalid UTF-8 string (specifically UTF-8) - 45 support for \eP, \ep, and \eX has not been compiled - 46 malformed \eP or \ep sequence - 47 unknown property name after \eP or \ep - 48 subpattern name is too long (maximum 32 characters) - 49 too many named subpatterns (maximum 10000) - 50 [this code is not in use] - 51 octal value is greater than \e377 in 8-bit non-UTF-8 mode - 52 internal error: overran compiling workspace - 53 internal error: previously-checked referenced subpattern - not found - 54 DEFINE group contains more than one branch - 55 repeating a DEFINE group is not allowed - 56 inconsistent NEWLINE options - 57 \eg is not followed by a braced, angle-bracketed, or quoted - name/number or by a plain number - 58 a numbered reference must not be zero - 59 an argument is not allowed for (*ACCEPT), (*FAIL), or (*COMMIT) - 60 (*VERB) not recognized or malformed - 61 number is too big - 62 subpattern name expected - 63 digit expected after (?+ - 64 ] is an invalid data character in JavaScript compatibility mode - 65 different names for subpatterns of the same number are - not allowed - 66 (*MARK) must have an argument - 67 this version of PCRE is not compiled with Unicode property - support - 68 \ec must be followed by an ASCII character - 69 \ek is not followed by a braced, angle-bracketed, or quoted name - 70 internal error: unknown opcode in find_fixedlength() - 71 \eN is not supported in a class - 72 too many forward references - 73 disallowed Unicode code point (>= 0xd800 && <= 0xdfff) - 74 invalid UTF-16 string (specifically UTF-16) - 75 name is too long in (*MARK), (*PRUNE), (*SKIP), or (*THEN) - 76 character value in \eu.... sequence is too large - 77 invalid UTF-32 string (specifically UTF-32) - 78 setting UTF is disabled by the application - 79 non-hex character in \ex{} (closing brace missing?) - 80 non-octal character in \eo{} (closing brace missing?) - 81 missing opening brace after \eo - 82 parentheses are too deeply nested - 83 invalid range in character class - 84 group name must start with a non-digit - 85 parentheses are too deeply nested (stack check) -.sp -The numbers 32 and 10000 in errors 48 and 49 are defaults; different values may -be used if the limits were changed when PCRE was built. -. -. -.\" HTML <a name="studyingapattern"></a> -.SH "STUDYING A PATTERN" -.rs -.sp -.nf -.B pcre_extra *pcre_study(const pcre *\fIcode\fP, int \fIoptions\fP, -.B " const char **\fIerrptr\fP);" -.fi -.PP -If a compiled pattern is going to be used several times, it is worth spending -more time analyzing it in order to speed up the time taken for matching. The -function \fBpcre_study()\fP takes a pointer to a compiled pattern as its first -argument. If studying the pattern produces additional information that will -help speed up matching, \fBpcre_study()\fP returns a pointer to a -\fBpcre_extra\fP block, in which the \fIstudy_data\fP field points to the -results of the study. -.P -The returned value from \fBpcre_study()\fP can be passed directly to -\fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP. However, a \fBpcre_extra\fP block -also contains other fields that can be set by the caller before the block is -passed; these are described -.\" HTML <a href="#extradata"> -.\" </a> -below -.\" -in the section on matching a pattern. -.P -If studying the pattern does not produce any useful information, -\fBpcre_study()\fP returns NULL by default. In that circumstance, if the -calling program wants to pass any of the other fields to \fBpcre_exec()\fP or -\fBpcre_dfa_exec()\fP, it must set up its own \fBpcre_extra\fP block. However, -if \fBpcre_study()\fP is called with the PCRE_STUDY_EXTRA_NEEDED option, it -returns a \fBpcre_extra\fP block even if studying did not find any additional -information. It may still return NULL, however, if an error occurs in -\fBpcre_study()\fP. -.P -The second argument of \fBpcre_study()\fP contains option bits. There are three -further options in addition to PCRE_STUDY_EXTRA_NEEDED: -.sp - PCRE_STUDY_JIT_COMPILE - PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE - PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE -.sp -If any of these are set, and the just-in-time compiler is available, the -pattern is further compiled into machine code that executes much faster than -the \fBpcre_exec()\fP interpretive matching function. If the just-in-time -compiler is not available, these options are ignored. All undefined bits in the -\fIoptions\fP argument must be zero. -.P -JIT compilation is a heavyweight optimization. It can take some time for -patterns to be analyzed, and for one-off matches and simple patterns the -benefit of faster execution might be offset by a much slower study time. -Not all patterns can be optimized by the JIT compiler. For those that cannot be -handled, matching automatically falls back to the \fBpcre_exec()\fP -interpreter. For more details, see the -.\" HREF -\fBpcrejit\fP -.\" -documentation. -.P -The third argument for \fBpcre_study()\fP is a pointer for an error message. If -studying succeeds (even if no data is returned), the variable it points to is -set to NULL. Otherwise it is set to point to a textual error message. This is a -static string that is part of the library. You must not try to free it. You -should test the error pointer for NULL after calling \fBpcre_study()\fP, to be -sure that it has run successfully. -.P -When you are finished with a pattern, you can free the memory used for the -study data by calling \fBpcre_free_study()\fP. This function was added to the -API for release 8.20. For earlier versions, the memory could be freed with -\fBpcre_free()\fP, just like the pattern itself. This will still work in cases -where JIT optimization is not used, but it is advisable to change to the new -function when convenient. -.P -This is a typical way in which \fBpcre_study\fP() is used (except that in a -real application there should be tests for errors): -.sp - int rc; - pcre *re; - pcre_extra *sd; - re = pcre_compile("pattern", 0, &error, &erroroffset, NULL); - sd = pcre_study( - re, /* result of pcre_compile() */ - 0, /* no options */ - &error); /* set to NULL or points to a message */ - rc = pcre_exec( /* see below for details of pcre_exec() options */ - re, sd, "subject", 7, 0, 0, ovector, 30); - ... - pcre_free_study(sd); - pcre_free(re); -.sp -Studying a pattern does two things: first, a lower bound for the length of -subject string that is needed to match the pattern is computed. This does not -mean that there are any strings of that length that match, but it does -guarantee that no shorter strings match. The value is used to avoid wasting -time by trying to match strings that are shorter than the lower bound. You can -find out the value in a calling program via the \fBpcre_fullinfo()\fP function. -.P -Studying a pattern is also useful for non-anchored patterns that do not have a -single fixed starting character. A bitmap of possible starting bytes is -created. This speeds up finding a position in the subject at which to start -matching. (In 16-bit mode, the bitmap is used for 16-bit values less than 256. -In 32-bit mode, the bitmap is used for 32-bit values less than 256.) -.P -These two optimizations apply to both \fBpcre_exec()\fP and -\fBpcre_dfa_exec()\fP, and the information is also used by the JIT compiler. -The optimizations can be disabled by setting the PCRE_NO_START_OPTIMIZE option. -You might want to do this if your pattern contains callouts or (*MARK) and you -want to make use of these facilities in cases where matching fails. -.P -PCRE_NO_START_OPTIMIZE can be specified at either compile time or execution -time. However, if PCRE_NO_START_OPTIMIZE is passed to \fBpcre_exec()\fP, (that -is, after any JIT compilation has happened) JIT execution is disabled. For JIT -execution to work with PCRE_NO_START_OPTIMIZE, the option must be set at -compile time. -.P -There is a longer discussion of PCRE_NO_START_OPTIMIZE -.\" HTML <a href="#execoptions"> -.\" </a> -below. -.\" -. -. -.\" HTML <a name="localesupport"></a> -.SH "LOCALE SUPPORT" -.rs -.sp -PCRE handles caseless matching, and determines whether characters are letters, -digits, or whatever, by reference to a set of tables, indexed by character -code point. When running in UTF-8 mode, or in the 16- or 32-bit libraries, this -applies only to characters with code points less than 256. By default, -higher-valued code points never match escapes such as \ew or \ed. However, if -PCRE is built with Unicode property support, all characters can be tested with -\ep and \eP, or, alternatively, the PCRE_UCP option can be set when a pattern -is compiled; this causes \ew and friends to use Unicode property support -instead of the built-in tables. -.P -The use of locales with Unicode is discouraged. If you are handling characters -with code points greater than 128, you should either use Unicode support, or -use locales, but not try to mix the two. -.P -PCRE contains an internal set of tables that are used when the final argument -of \fBpcre_compile()\fP is NULL. These are sufficient for many applications. -Normally, the internal tables recognize only ASCII characters. However, when -PCRE is built, it is possible to cause the internal tables to be rebuilt in the -default "C" locale of the local system, which may cause them to be different. -.P -The internal tables can always be overridden by tables supplied by the -application that calls PCRE. These may be created in a different locale from -the default. As more and more applications change to using Unicode, the need -for this locale support is expected to die away. -.P -External tables are built by calling the \fBpcre_maketables()\fP function, -which has no arguments, in the relevant locale. The result can then be passed -to \fBpcre_compile()\fP as often as necessary. For example, to build and use -tables that are appropriate for the French locale (where accented characters -with values greater than 128 are treated as letters), the following code could -be used: -.sp - setlocale(LC_CTYPE, "fr_FR"); - tables = pcre_maketables(); - re = pcre_compile(..., tables); -.sp -The locale name "fr_FR" is used on Linux and other Unix-like systems; if you -are using Windows, the name for the French locale is "french". -.P -When \fBpcre_maketables()\fP runs, the tables are built in memory that is -obtained via \fBpcre_malloc\fP. It is the caller's responsibility to ensure -that the memory containing the tables remains available for as long as it is -needed. -.P -The pointer that is passed to \fBpcre_compile()\fP is saved with the compiled -pattern, and the same tables are used via this pointer by \fBpcre_study()\fP -and also by \fBpcre_exec()\fP and \fBpcre_dfa_exec()\fP. Thus, for any single -pattern, compilation, studying and matching all happen in the same locale, but -different patterns can be processed in different locales. -.P -It is possible to pass a table pointer or NULL (indicating the use of the -internal tables) to \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP (see the -discussion below in the section on matching a pattern). This facility is -provided for use with pre-compiled patterns that have been saved and reloaded. -Character tables are not saved with patterns, so if a non-standard table was -used at compile time, it must be provided again when the reloaded pattern is -matched. Attempting to use this facility to match a pattern in a different -locale from the one in which it was compiled is likely to lead to anomalous -(usually incorrect) results. -. -. -.\" HTML <a name="infoaboutpattern"></a> -.SH "INFORMATION ABOUT A PATTERN" -.rs -.sp -.nf -.B int pcre_fullinfo(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," -.B " int \fIwhat\fP, void *\fIwhere\fP);" -.fi -.PP -The \fBpcre_fullinfo()\fP function returns information about a compiled -pattern. It replaces the \fBpcre_info()\fP function, which was removed from the -library at version 8.30, after more than 10 years of obsolescence. -.P -The first argument for \fBpcre_fullinfo()\fP is a pointer to the compiled -pattern. The second argument is the result of \fBpcre_study()\fP, or NULL if -the pattern was not studied. The third argument specifies which piece of -information is required, and the fourth argument is a pointer to a variable -to receive the data. The yield of the function is zero for success, or one of -the following negative numbers: -.sp - PCRE_ERROR_NULL the argument \fIcode\fP was NULL - the argument \fIwhere\fP was NULL - PCRE_ERROR_BADMAGIC the "magic number" was not found - PCRE_ERROR_BADENDIANNESS the pattern was compiled with different - endianness - PCRE_ERROR_BADOPTION the value of \fIwhat\fP was invalid - PCRE_ERROR_UNSET the requested field is not set -.sp -The "magic number" is placed at the start of each compiled pattern as an simple -check against passing an arbitrary memory pointer. The endianness error can -occur if a compiled pattern is saved and reloaded on a different host. Here is -a typical call of \fBpcre_fullinfo()\fP, to obtain the length of the compiled -pattern: -.sp - int rc; - size_t length; - rc = pcre_fullinfo( - re, /* result of pcre_compile() */ - sd, /* result of pcre_study(), or NULL */ - PCRE_INFO_SIZE, /* what is required */ - &length); /* where to put the data */ -.sp -The possible values for the third argument are defined in \fBpcre.h\fP, and are -as follows: -.sp - PCRE_INFO_BACKREFMAX -.sp -Return the number of the highest back reference in the pattern. The fourth -argument should point to an \fBint\fP variable. Zero is returned if there are -no back references. -.sp - PCRE_INFO_CAPTURECOUNT -.sp -Return the number of capturing subpatterns in the pattern. The fourth argument -should point to an \fBint\fP variable. -.sp - PCRE_INFO_DEFAULT_TABLES -.sp -Return a pointer to the internal default character tables within PCRE. The -fourth argument should point to an \fBunsigned char *\fP variable. This -information call is provided for internal use by the \fBpcre_study()\fP -function. External callers can cause PCRE to use its internal tables by passing -a NULL table pointer. -.sp - PCRE_INFO_FIRSTBYTE (deprecated) -.sp -Return information about the first data unit of any matched string, for a -non-anchored pattern. The name of this option refers to the 8-bit library, -where data units are bytes. The fourth argument should point to an \fBint\fP -variable. Negative values are used for special cases. However, this means that -when the 32-bit library is in non-UTF-32 mode, the full 32-bit range of -characters cannot be returned. For this reason, this value is deprecated; use -PCRE_INFO_FIRSTCHARACTERFLAGS and PCRE_INFO_FIRSTCHARACTER instead. -.P -If there is a fixed first value, for example, the letter "c" from a pattern -such as (cat|cow|coyote), its value is returned. In the 8-bit library, the -value is always less than 256. In the 16-bit library the value can be up to -0xffff. In the 32-bit library the value can be up to 0x10ffff. -.P -If there is no fixed first value, and if either -.sp -(a) the pattern was compiled with the PCRE_MULTILINE option, and every branch -starts with "^", or -.sp -(b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not set -(if it were set, the pattern would be anchored), -.sp --1 is returned, indicating that the pattern matches only at the start of a -subject string or after any newline within the string. Otherwise -2 is -returned. For anchored patterns, -2 is returned. -.sp - PCRE_INFO_FIRSTCHARACTER -.sp -Return the value of the first data unit (non-UTF character) of any matched -string in the situation where PCRE_INFO_FIRSTCHARACTERFLAGS returns 1; -otherwise return 0. The fourth argument should point to an \fBuint_t\fP -variable. -.P -In the 8-bit library, the value is always less than 256. In the 16-bit library -the value can be up to 0xffff. In the 32-bit library in UTF-32 mode the value -can be up to 0x10ffff, and up to 0xffffffff when not using UTF-32 mode. -.sp - PCRE_INFO_FIRSTCHARACTERFLAGS -.sp -Return information about the first data unit of any matched string, for a -non-anchored pattern. The fourth argument should point to an \fBint\fP -variable. -.P -If there is a fixed first value, for example, the letter "c" from a pattern -such as (cat|cow|coyote), 1 is returned, and the character value can be -retrieved using PCRE_INFO_FIRSTCHARACTER. If there is no fixed first value, and -if either -.sp -(a) the pattern was compiled with the PCRE_MULTILINE option, and every branch -starts with "^", or -.sp -(b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not set -(if it were set, the pattern would be anchored), -.sp -2 is returned, indicating that the pattern matches only at the start of a -subject string or after any newline within the string. Otherwise 0 is -returned. For anchored patterns, 0 is returned. -.sp - PCRE_INFO_FIRSTTABLE -.sp -If the pattern was studied, and this resulted in the construction of a 256-bit -table indicating a fixed set of values for the first data unit in any matching -string, a pointer to the table is returned. Otherwise NULL is returned. The -fourth argument should point to an \fBunsigned char *\fP variable. -.sp - PCRE_INFO_HASCRORLF -.sp -Return 1 if the pattern contains any explicit matches for CR or LF characters, -otherwise 0. The fourth argument should point to an \fBint\fP variable. An -explicit match is either a literal CR or LF character, or \er or \en. -.sp - PCRE_INFO_JCHANGED -.sp -Return 1 if the (?J) or (?-J) option setting is used in the pattern, otherwise -0. The fourth argument should point to an \fBint\fP variable. (?J) and -(?-J) set and unset the local PCRE_DUPNAMES option, respectively. -.sp - PCRE_INFO_JIT -.sp -Return 1 if the pattern was studied with one of the JIT options, and -just-in-time compiling was successful. The fourth argument should point to an -\fBint\fP variable. A return value of 0 means that JIT support is not available -in this version of PCRE, or that the pattern was not studied with a JIT option, -or that the JIT compiler could not handle this particular pattern. See the -.\" HREF -\fBpcrejit\fP -.\" -documentation for details of what can and cannot be handled. -.sp - PCRE_INFO_JITSIZE -.sp -If the pattern was successfully studied with a JIT option, return the size of -the JIT compiled code, otherwise return zero. The fourth argument should point -to a \fBsize_t\fP variable. -.sp - PCRE_INFO_LASTLITERAL -.sp -Return the value of the rightmost literal data unit that must exist in any -matched string, other than at its start, if such a value has been recorded. The -fourth argument should point to an \fBint\fP variable. If there is no such -value, -1 is returned. For anchored patterns, a last literal value is recorded -only if it follows something of variable length. For example, for the pattern -/^a\ed+z\ed+/ the returned value is "z", but for /^a\edz\ed/ the returned value -is -1. -.P -Since for the 32-bit library using the non-UTF-32 mode, this function is unable -to return the full 32-bit range of characters, this value is deprecated; -instead the PCRE_INFO_REQUIREDCHARFLAGS and PCRE_INFO_REQUIREDCHAR values should -be used. -.sp - PCRE_INFO_MATCH_EMPTY -.sp -Return 1 if the pattern can match an empty string, otherwise 0. The fourth -argument should point to an \fBint\fP variable. -.sp - PCRE_INFO_MATCHLIMIT -.sp -If the pattern set a match limit by including an item of the form -(*LIMIT_MATCH=nnnn) at the start, the value is returned. The fourth argument -should point to an unsigned 32-bit integer. If no such value has been set, the -call to \fBpcre_fullinfo()\fP returns the error PCRE_ERROR_UNSET. -.sp - PCRE_INFO_MAXLOOKBEHIND -.sp -Return the number of characters (NB not data units) in the longest lookbehind -assertion in the pattern. This information is useful when doing multi-segment -matching using the partial matching facilities. Note that the simple assertions -\eb and \eB require a one-character lookbehind. \eA also registers a -one-character lookbehind, though it does not actually inspect the previous -character. This is to ensure that at least one character from the old segment -is retained when a new segment is processed. Otherwise, if there are no -lookbehinds in the pattern, \eA might match incorrectly at the start of a new -segment. -.sp - PCRE_INFO_MINLENGTH -.sp -If the pattern was studied and a minimum length for matching subject strings -was computed, its value is returned. Otherwise the returned value is -1. The -value is a number of characters, which in UTF mode may be different from the -number of data units. The fourth argument should point to an \fBint\fP -variable. A non-negative value is a lower bound to the length of any matching -string. There may not be any strings of that length that do actually match, but -every string that does match is at least that long. -.sp - PCRE_INFO_NAMECOUNT - PCRE_INFO_NAMEENTRYSIZE - PCRE_INFO_NAMETABLE -.sp -PCRE supports the use of named as well as numbered capturing parentheses. The -names are just an additional way of identifying the parentheses, which still -acquire numbers. Several convenience functions such as -\fBpcre_get_named_substring()\fP are provided for extracting captured -substrings by name. It is also possible to extract the data directly, by first -converting the name to a number in order to access the correct pointers in the -output vector (described with \fBpcre_exec()\fP below). To do the conversion, -you need to use the name-to-number map, which is described by these three -values. -.P -The map consists of a number of fixed-size entries. PCRE_INFO_NAMECOUNT gives -the number of entries, and PCRE_INFO_NAMEENTRYSIZE gives the size of each -entry; both of these return an \fBint\fP value. The entry size depends on the -length of the longest name. PCRE_INFO_NAMETABLE returns a pointer to the first -entry of the table. This is a pointer to \fBchar\fP in the 8-bit library, where -the first two bytes of each entry are the number of the capturing parenthesis, -most significant byte first. In the 16-bit library, the pointer points to -16-bit data units, the first of which contains the parenthesis number. In the -32-bit library, the pointer points to 32-bit data units, the first of which -contains the parenthesis number. The rest of the entry is the corresponding -name, zero terminated. -.P -The names are in alphabetical order. If (?| is used to create multiple groups -with the same number, as described in the -.\" HTML <a href="pcrepattern.html#dupsubpatternnumber"> -.\" </a> -section on duplicate subpattern numbers -.\" -in the -.\" HREF -\fBpcrepattern\fP -.\" -page, the groups may be given the same name, but there is only one entry in the -table. Different names for groups of the same number are not permitted. -Duplicate names for subpatterns with different numbers are permitted, -but only if PCRE_DUPNAMES is set. They appear in the table in the order in -which they were found in the pattern. In the absence of (?| this is the order -of increasing number; when (?| is used this is not necessarily the case because -later subpatterns may have lower numbers. -.P -As a simple example of the name/number table, consider the following pattern -after compilation by the 8-bit library (assume PCRE_EXTENDED is set, so white -space - including newlines - is ignored): -.sp -.\" JOIN - (?<date> (?<year>(\ed\ed)?\ed\ed) - - (?<month>\ed\ed) - (?<day>\ed\ed) ) -.sp -There are four named subpatterns, so the table has four entries, and each entry -in the table is eight bytes long. The table is as follows, with non-printing -bytes shows in hexadecimal, and undefined bytes shown as ??: -.sp - 00 01 d a t e 00 ?? - 00 05 d a y 00 ?? ?? - 00 04 m o n t h 00 - 00 02 y e a r 00 ?? -.sp -When writing code to extract data from named subpatterns using the -name-to-number map, remember that the length of the entries is likely to be -different for each compiled pattern. -.sp - PCRE_INFO_OKPARTIAL -.sp -Return 1 if the pattern can be used for partial matching with -\fBpcre_exec()\fP, otherwise 0. The fourth argument should point to an -\fBint\fP variable. From release 8.00, this always returns 1, because the -restrictions that previously applied to partial matching have been lifted. The -.\" HREF -\fBpcrepartial\fP -.\" -documentation gives details of partial matching. -.sp - PCRE_INFO_OPTIONS -.sp -Return a copy of the options with which the pattern was compiled. The fourth -argument should point to an \fBunsigned long int\fP variable. These option bits -are those specified in the call to \fBpcre_compile()\fP, modified by any -top-level option settings at the start of the pattern itself. In other words, -they are the options that will be in force when matching starts. For example, -if the pattern /(?im)abc(?-i)d/ is compiled with the PCRE_EXTENDED option, the -result is PCRE_CASELESS, PCRE_MULTILINE, and PCRE_EXTENDED. -.P -A pattern is automatically anchored by PCRE if all of its top-level -alternatives begin with one of the following: -.sp - ^ unless PCRE_MULTILINE is set - \eA always - \eG always -.\" JOIN - .* if PCRE_DOTALL is set and there are no back - references to the subpattern in which .* appears -.sp -For such patterns, the PCRE_ANCHORED bit is set in the options returned by -\fBpcre_fullinfo()\fP. -.sp - PCRE_INFO_RECURSIONLIMIT -.sp -If the pattern set a recursion limit by including an item of the form -(*LIMIT_RECURSION=nnnn) at the start, the value is returned. The fourth -argument should point to an unsigned 32-bit integer. If no such value has been -set, the call to \fBpcre_fullinfo()\fP returns the error PCRE_ERROR_UNSET. -.sp - PCRE_INFO_SIZE -.sp -Return the size of the compiled pattern in bytes (for all three libraries). The -fourth argument should point to a \fBsize_t\fP variable. This value does not -include the size of the \fBpcre\fP structure that is returned by -\fBpcre_compile()\fP. The value that is passed as the argument to -\fBpcre_malloc()\fP when \fBpcre_compile()\fP is getting memory in which to -place the compiled data is the value returned by this option plus the size of -the \fBpcre\fP structure. Studying a compiled pattern, with or without JIT, -does not alter the value returned by this option. -.sp - PCRE_INFO_STUDYSIZE -.sp -Return the size in bytes (for all three libraries) of the data block pointed to -by the \fIstudy_data\fP field in a \fBpcre_extra\fP block. If \fBpcre_extra\fP -is NULL, or there is no study data, zero is returned. The fourth argument -should point to a \fBsize_t\fP variable. The \fIstudy_data\fP field is set by -\fBpcre_study()\fP to record information that will speed up matching (see the -section entitled -.\" HTML <a href="#studyingapattern"> -.\" </a> -"Studying a pattern" -.\" -above). The format of the \fIstudy_data\fP block is private, but its length -is made available via this option so that it can be saved and restored (see the -.\" HREF -\fBpcreprecompile\fP -.\" -documentation for details). -.sp - PCRE_INFO_REQUIREDCHARFLAGS -.sp -Returns 1 if there is a rightmost literal data unit that must exist in any -matched string, other than at its start. The fourth argument should point to -an \fBint\fP variable. If there is no such value, 0 is returned. If returning -1, the character value itself can be retrieved using PCRE_INFO_REQUIREDCHAR. -.P -For anchored patterns, a last literal value is recorded only if it follows -something of variable length. For example, for the pattern /^a\ed+z\ed+/ the -returned value 1 (with "z" returned from PCRE_INFO_REQUIREDCHAR), but for -/^a\edz\ed/ the returned value is 0. -.sp - PCRE_INFO_REQUIREDCHAR -.sp -Return the value of the rightmost literal data unit that must exist in any -matched string, other than at its start, if such a value has been recorded. The -fourth argument should point to an \fBuint32_t\fP variable. If there is no such -value, 0 is returned. -. -. -.SH "REFERENCE COUNTS" -.rs -.sp -.B int pcre_refcount(pcre *\fIcode\fP, int \fIadjust\fP); -.PP -The \fBpcre_refcount()\fP function is used to maintain a reference count in the -data block that contains a compiled pattern. It is provided for the benefit of -applications that operate in an object-oriented manner, where different parts -of the application may be using the same compiled pattern, but you want to free -the block when they are all done. -.P -When a pattern is compiled, the reference count field is initialized to zero. -It is changed only by calling this function, whose action is to add the -\fIadjust\fP value (which may be positive or negative) to it. The yield of the -function is the new value. However, the value of the count is constrained to -lie between 0 and 65535, inclusive. If the new value is outside these limits, -it is forced to the appropriate limit value. -.P -Except when it is zero, the reference count is not correctly preserved if a -pattern is compiled on one host and then transferred to a host whose byte-order -is different. (This seems a highly unlikely scenario.) -. -. -.SH "MATCHING A PATTERN: THE TRADITIONAL FUNCTION" -.rs -.sp -.nf -.B int pcre_exec(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," -.B " const char *\fIsubject\fP," int \fIlength\fP, int \fIstartoffset\fP, -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP);" -.fi -.P -The function \fBpcre_exec()\fP is called to match a subject string against a -compiled pattern, which is passed in the \fIcode\fP argument. If the -pattern was studied, the result of the study should be passed in the -\fIextra\fP argument. You can call \fBpcre_exec()\fP with the same \fIcode\fP -and \fIextra\fP arguments as many times as you like, in order to match -different subject strings with the same pattern. -.P -This function is the main matching facility of the library, and it operates in -a Perl-like manner. For specialist use there is also an alternative matching -function, which is described -.\" HTML <a href="#dfamatch"> -.\" </a> -below -.\" -in the section about the \fBpcre_dfa_exec()\fP function. -.P -In most applications, the pattern will have been compiled (and optionally -studied) in the same process that calls \fBpcre_exec()\fP. However, it is -possible to save compiled patterns and study data, and then use them later -in different processes, possibly even on different hosts. For a discussion -about this, see the -.\" HREF -\fBpcreprecompile\fP -.\" -documentation. -.P -Here is an example of a simple call to \fBpcre_exec()\fP: -.sp - int rc; - int ovector[30]; - rc = pcre_exec( - re, /* result of pcre_compile() */ - NULL, /* we didn't study the pattern */ - "some string", /* the subject string */ - 11, /* the length of the subject string */ - 0, /* start at offset 0 in the subject */ - 0, /* default options */ - ovector, /* vector of integers for substring information */ - 30); /* number of elements (NOT size in bytes) */ -. -. -.\" HTML <a name="extradata"></a> -.SS "Extra data for \fBpcre_exec()\fR" -.rs -.sp -If the \fIextra\fP argument is not NULL, it must point to a \fBpcre_extra\fP -data block. The \fBpcre_study()\fP function returns such a block (when it -doesn't return NULL), but you can also create one for yourself, and pass -additional information in it. The \fBpcre_extra\fP block contains the following -fields (not necessarily in this order): -.sp - unsigned long int \fIflags\fP; - void *\fIstudy_data\fP; - void *\fIexecutable_jit\fP; - unsigned long int \fImatch_limit\fP; - unsigned long int \fImatch_limit_recursion\fP; - void *\fIcallout_data\fP; - const unsigned char *\fItables\fP; - unsigned char **\fImark\fP; -.sp -In the 16-bit version of this structure, the \fImark\fP field has type -"PCRE_UCHAR16 **". -.sp -In the 32-bit version of this structure, the \fImark\fP field has type -"PCRE_UCHAR32 **". -.P -The \fIflags\fP field is used to specify which of the other fields are set. The -flag bits are: -.sp - PCRE_EXTRA_CALLOUT_DATA - PCRE_EXTRA_EXECUTABLE_JIT - PCRE_EXTRA_MARK - PCRE_EXTRA_MATCH_LIMIT - PCRE_EXTRA_MATCH_LIMIT_RECURSION - PCRE_EXTRA_STUDY_DATA - PCRE_EXTRA_TABLES -.sp -Other flag bits should be set to zero. The \fIstudy_data\fP field and sometimes -the \fIexecutable_jit\fP field are set in the \fBpcre_extra\fP block that is -returned by \fBpcre_study()\fP, together with the appropriate flag bits. You -should not set these yourself, but you may add to the block by setting other -fields and their corresponding flag bits. -.P -The \fImatch_limit\fP field provides a means of preventing PCRE from using up a -vast amount of resources when running patterns that are not going to match, -but which have a very large number of possibilities in their search trees. The -classic example is a pattern that uses nested unlimited repeats. -.P -Internally, \fBpcre_exec()\fP uses a function called \fBmatch()\fP, which it -calls repeatedly (sometimes recursively). The limit set by \fImatch_limit\fP is -imposed on the number of times this function is called during a match, which -has the effect of limiting the amount of backtracking that can take place. For -patterns that are not anchored, the count restarts from zero for each position -in the subject string. -.P -When \fBpcre_exec()\fP is called with a pattern that was successfully studied -with a JIT option, the way that the matching is executed is entirely different. -However, there is still the possibility of runaway matching that goes on for a -very long time, and so the \fImatch_limit\fP value is also used in this case -(but in a different way) to limit how long the matching can continue. -.P -The default value for the limit can be set when PCRE is built; the default -default is 10 million, which handles all but the most extreme cases. You can -override the default by suppling \fBpcre_exec()\fP with a \fBpcre_extra\fP -block in which \fImatch_limit\fP is set, and PCRE_EXTRA_MATCH_LIMIT is set in -the \fIflags\fP field. If the limit is exceeded, \fBpcre_exec()\fP returns -PCRE_ERROR_MATCHLIMIT. -.P -A value for the match limit may also be supplied by an item at the start of a -pattern of the form -.sp - (*LIMIT_MATCH=d) -.sp -where d is a decimal number. However, such a setting is ignored unless d is -less than the limit set by the caller of \fBpcre_exec()\fP or, if no such limit -is set, less than the default. -.P -The \fImatch_limit_recursion\fP field is similar to \fImatch_limit\fP, but -instead of limiting the total number of times that \fBmatch()\fP is called, it -limits the depth of recursion. The recursion depth is a smaller number than the -total number of calls, because not all calls to \fBmatch()\fP are recursive. -This limit is of use only if it is set smaller than \fImatch_limit\fP. -.P -Limiting the recursion depth limits the amount of machine stack that can be -used, or, when PCRE has been compiled to use memory on the heap instead of the -stack, the amount of heap memory that can be used. This limit is not relevant, -and is ignored, when matching is done using JIT compiled code. -.P -The default value for \fImatch_limit_recursion\fP can be set when PCRE is -built; the default default is the same value as the default for -\fImatch_limit\fP. You can override the default by suppling \fBpcre_exec()\fP -with a \fBpcre_extra\fP block in which \fImatch_limit_recursion\fP is set, and -PCRE_EXTRA_MATCH_LIMIT_RECURSION is set in the \fIflags\fP field. If the limit -is exceeded, \fBpcre_exec()\fP returns PCRE_ERROR_RECURSIONLIMIT. -.P -A value for the recursion limit may also be supplied by an item at the start of -a pattern of the form -.sp - (*LIMIT_RECURSION=d) -.sp -where d is a decimal number. However, such a setting is ignored unless d is -less than the limit set by the caller of \fBpcre_exec()\fP or, if no such limit -is set, less than the default. -.P -The \fIcallout_data\fP field is used in conjunction with the "callout" feature, -and is described in the -.\" HREF -\fBpcrecallout\fP -.\" -documentation. -.P -The \fItables\fP field is provided for use with patterns that have been -pre-compiled using custom character tables, saved to disc or elsewhere, and -then reloaded, because the tables that were used to compile a pattern are not -saved with it. See the -.\" HREF -\fBpcreprecompile\fP -.\" -documentation for a discussion of saving compiled patterns for later use. If -NULL is passed using this mechanism, it forces PCRE's internal tables to be -used. -.P -\fBWarning:\fP The tables that \fBpcre_exec()\fP uses must be the same as those -that were used when the pattern was compiled. If this is not the case, the -behaviour of \fBpcre_exec()\fP is undefined. Therefore, when a pattern is -compiled and matched in the same process, this field should never be set. In -this (the most common) case, the correct table pointer is automatically passed -with the compiled pattern from \fBpcre_compile()\fP to \fBpcre_exec()\fP. -.P -If PCRE_EXTRA_MARK is set in the \fIflags\fP field, the \fImark\fP field must -be set to point to a suitable variable. If the pattern contains any -backtracking control verbs such as (*MARK:NAME), and the execution ends up with -a name to pass back, a pointer to the name string (zero terminated) is placed -in the variable pointed to by the \fImark\fP field. The names are within the -compiled pattern; if you wish to retain such a name you must copy it before -freeing the memory of a compiled pattern. If there is no name to pass back, the -variable pointed to by the \fImark\fP field is set to NULL. For details of the -backtracking control verbs, see the section entitled -.\" HTML <a href="pcrepattern#backtrackcontrol"> -.\" </a> -"Backtracking control" -.\" -in the -.\" HREF -\fBpcrepattern\fP -.\" -documentation. -. -. -.\" HTML <a name="execoptions"></a> -.SS "Option bits for \fBpcre_exec()\fP" -.rs -.sp -The unused bits of the \fIoptions\fP argument for \fBpcre_exec()\fP must be -zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_\fIxxx\fP, -PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, -PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_HARD, and -PCRE_PARTIAL_SOFT. -.P -If the pattern was successfully studied with one of the just-in-time (JIT) -compile options, the only supported options for JIT execution are -PCRE_NO_UTF8_CHECK, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, -PCRE_NOTEMPTY_ATSTART, PCRE_PARTIAL_HARD, and PCRE_PARTIAL_SOFT. If an -unsupported option is used, JIT execution is disabled and the normal -interpretive code in \fBpcre_exec()\fP is run. -.sp - PCRE_ANCHORED -.sp -The PCRE_ANCHORED option limits \fBpcre_exec()\fP to matching at the first -matching position. If a pattern was compiled with PCRE_ANCHORED, or turned out -to be anchored by virtue of its contents, it cannot be made unachored at -matching time. -.sp - PCRE_BSR_ANYCRLF - PCRE_BSR_UNICODE -.sp -These options (which are mutually exclusive) control what the \eR escape -sequence matches. The choice is either to match only CR, LF, or CRLF, or to -match any Unicode newline sequence. These options override the choice that was -made or defaulted when the pattern was compiled. -.sp - PCRE_NEWLINE_CR - PCRE_NEWLINE_LF - PCRE_NEWLINE_CRLF - PCRE_NEWLINE_ANYCRLF - PCRE_NEWLINE_ANY -.sp -These options override the newline definition that was chosen or defaulted when -the pattern was compiled. For details, see the description of -\fBpcre_compile()\fP above. During matching, the newline choice affects the -behaviour of the dot, circumflex, and dollar metacharacters. It may also alter -the way the match position is advanced after a match failure for an unanchored -pattern. -.P -When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is set, and a -match attempt for an unanchored pattern fails when the current position is at a -CRLF sequence, and the pattern contains no explicit matches for CR or LF -characters, the match position is advanced by two characters instead of one, in -other words, to after the CRLF. -.P -The above rule is a compromise that makes the most common cases work as -expected. For example, if the pattern is .+A (and the PCRE_DOTALL option is not -set), it does not match the string "\er\enA" because, after failing at the -start, it skips both the CR and the LF before retrying. However, the pattern -[\er\en]A does match that string, because it contains an explicit CR or LF -reference, and so advances only by one character after the first failure. -.P -An explicit match for CR of LF is either a literal appearance of one of those -characters, or one of the \er or \en escape sequences. Implicit matches such as -[^X] do not count, nor does \es (which includes CR and LF in the characters -that it matches). -.P -Notwithstanding the above, anomalous effects may still occur when CRLF is a -valid newline sequence and explicit \er or \en escapes appear in the pattern. -.sp - PCRE_NOTBOL -.sp -This option specifies that first character of the subject string is not the -beginning of a line, so the circumflex metacharacter should not match before -it. Setting this without PCRE_MULTILINE (at compile time) causes circumflex -never to match. This option affects only the behaviour of the circumflex -metacharacter. It does not affect \eA. -.sp - PCRE_NOTEOL -.sp -This option specifies that the end of the subject string is not the end of a -line, so the dollar metacharacter should not match it nor (except in multiline -mode) a newline immediately before it. Setting this without PCRE_MULTILINE (at -compile time) causes dollar never to match. This option affects only the -behaviour of the dollar metacharacter. It does not affect \eZ or \ez. -.sp - PCRE_NOTEMPTY -.sp -An empty string is not considered to be a valid match if this option is set. If -there are alternatives in the pattern, they are tried. If all the alternatives -match the empty string, the entire match fails. For example, if the pattern -.sp - a?b? -.sp -is applied to a string not beginning with "a" or "b", it matches an empty -string at the start of the subject. With PCRE_NOTEMPTY set, this match is not -valid, so PCRE searches further into the string for occurrences of "a" or "b". -.sp - PCRE_NOTEMPTY_ATSTART -.sp -This is like PCRE_NOTEMPTY, except that an empty string match that is not at -the start of the subject is permitted. If the pattern is anchored, such a match -can occur only if the pattern contains \eK. -.P -Perl has no direct equivalent of PCRE_NOTEMPTY or PCRE_NOTEMPTY_ATSTART, but it -does make a special case of a pattern match of the empty string within its -\fBsplit()\fP function, and when using the /g modifier. It is possible to -emulate Perl's behaviour after matching a null string by first trying the match -again at the same offset with PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED, and then -if that fails, by advancing the starting offset (see below) and trying an -ordinary match again. There is some code that demonstrates how to do this in -the -.\" HREF -\fBpcredemo\fP -.\" -sample program. In the most general case, you have to check to see if the -newline convention recognizes CRLF as a newline, and if so, and the current -character is CR followed by LF, advance the starting offset by two characters -instead of one. -.sp - PCRE_NO_START_OPTIMIZE -.sp -There are a number of optimizations that \fBpcre_exec()\fP uses at the start of -a match, in order to speed up the process. For example, if it is known that an -unanchored match must start with a specific character, it searches the subject -for that character, and fails immediately if it cannot find it, without -actually running the main matching function. This means that a special item -such as (*COMMIT) at the start of a pattern is not considered until after a -suitable starting point for the match has been found. Also, when callouts or -(*MARK) items are in use, these "start-up" optimizations can cause them to be -skipped if the pattern is never actually used. The start-up optimizations are -in effect a pre-scan of the subject that takes place before the pattern is run. -.P -The PCRE_NO_START_OPTIMIZE option disables the start-up optimizations, possibly -causing performance to suffer, but ensuring that in cases where the result is -"no match", the callouts do occur, and that items such as (*COMMIT) and (*MARK) -are considered at every possible starting position in the subject string. If -PCRE_NO_START_OPTIMIZE is set at compile time, it cannot be unset at matching -time. The use of PCRE_NO_START_OPTIMIZE at matching time (that is, passing it -to \fBpcre_exec()\fP) disables JIT execution; in this situation, matching is -always done using interpretively. -.P -Setting PCRE_NO_START_OPTIMIZE can change the outcome of a matching operation. -Consider the pattern -.sp - (*COMMIT)ABC -.sp -When this is compiled, PCRE records the fact that a match must start with the -character "A". Suppose the subject string is "DEFABC". The start-up -optimization scans along the subject, finds "A" and runs the first match -attempt from there. The (*COMMIT) item means that the pattern must match the -current starting position, which in this case, it does. However, if the same -match is run with PCRE_NO_START_OPTIMIZE set, the initial scan along the -subject string does not happen. The first match attempt is run starting from -"D" and when this fails, (*COMMIT) prevents any further matches being tried, so -the overall result is "no match". If the pattern is studied, more start-up -optimizations may be used. For example, a minimum length for the subject may be -recorded. Consider the pattern -.sp - (*MARK:A)(X|Y) -.sp -The minimum length for a match is one character. If the subject is "ABC", there -will be attempts to match "ABC", "BC", "C", and then finally an empty string. -If the pattern is studied, the final attempt does not take place, because PCRE -knows that the subject is too short, and so the (*MARK) is never encountered. -In this case, studying the pattern does not affect the overall match result, -which is still "no match", but it does affect the auxiliary information that is -returned. -.sp - PCRE_NO_UTF8_CHECK -.sp -When PCRE_UTF8 is set at compile time, the validity of the subject as a UTF-8 -string is automatically checked when \fBpcre_exec()\fP is subsequently called. -The entire string is checked before any other processing takes place. The value -of \fIstartoffset\fP is also checked to ensure that it points to the start of a -UTF-8 character. There is a discussion about the -.\" HTML <a href="pcreunicode.html#utf8strings"> -.\" </a> -validity of UTF-8 strings -.\" -in the -.\" HREF -\fBpcreunicode\fP -.\" -page. If an invalid sequence of bytes is found, \fBpcre_exec()\fP returns the -error PCRE_ERROR_BADUTF8 or, if PCRE_PARTIAL_HARD is set and the problem is a -truncated character at the end of the subject, PCRE_ERROR_SHORTUTF8. In both -cases, information about the precise nature of the error may also be returned -(see the descriptions of these errors in the section entitled \fIError return -values from\fP \fBpcre_exec()\fP -.\" HTML <a href="#errorlist"> -.\" </a> -below). -.\" -If \fIstartoffset\fP contains a value that does not point to the start of a -UTF-8 character (or to the end of the subject), PCRE_ERROR_BADUTF8_OFFSET is -returned. -.P -If you already know that your subject is valid, and you want to skip these -checks for performance reasons, you can set the PCRE_NO_UTF8_CHECK option when -calling \fBpcre_exec()\fP. You might want to do this for the second and -subsequent calls to \fBpcre_exec()\fP if you are making repeated calls to find -all the matches in a single subject string. However, you should be sure that -the value of \fIstartoffset\fP points to the start of a character (or the end -of the subject). When PCRE_NO_UTF8_CHECK is set, the effect of passing an -invalid string as a subject or an invalid value of \fIstartoffset\fP is -undefined. Your program may crash or loop. -.sp - PCRE_PARTIAL_HARD - PCRE_PARTIAL_SOFT -.sp -These options turn on the partial matching feature. For backwards -compatibility, PCRE_PARTIAL is a synonym for PCRE_PARTIAL_SOFT. A partial match -occurs if the end of the subject string is reached successfully, but there are -not enough subject characters to complete the match. If this happens when -PCRE_PARTIAL_SOFT (but not PCRE_PARTIAL_HARD) is set, matching continues by -testing any remaining alternatives. Only if no complete match can be found is -PCRE_ERROR_PARTIAL returned instead of PCRE_ERROR_NOMATCH. In other words, -PCRE_PARTIAL_SOFT says that the caller is prepared to handle a partial match, -but only if no complete match can be found. -.P -If PCRE_PARTIAL_HARD is set, it overrides PCRE_PARTIAL_SOFT. In this case, if a -partial match is found, \fBpcre_exec()\fP immediately returns -PCRE_ERROR_PARTIAL, without considering any other alternatives. In other words, -when PCRE_PARTIAL_HARD is set, a partial match is considered to be more -important that an alternative complete match. -.P -In both cases, the portion of the string that was inspected when the partial -match was found is set as the first matching string. There is a more detailed -discussion of partial and multi-segment matching, with examples, in the -.\" HREF -\fBpcrepartial\fP -.\" -documentation. -. -. -.SS "The string to be matched by \fBpcre_exec()\fP" -.rs -.sp -The subject string is passed to \fBpcre_exec()\fP as a pointer in -\fIsubject\fP, a length in \fIlength\fP, and a starting offset in -\fIstartoffset\fP. The units for \fIlength\fP and \fIstartoffset\fP are bytes -for the 8-bit library, 16-bit data items for the 16-bit library, and 32-bit -data items for the 32-bit library. -.P -If \fIstartoffset\fP is negative or greater than the length of the subject, -\fBpcre_exec()\fP returns PCRE_ERROR_BADOFFSET. When the starting offset is -zero, the search for a match starts at the beginning of the subject, and this -is by far the most common case. In UTF-8 or UTF-16 mode, the offset must point -to the start of a character, or the end of the subject (in UTF-32 mode, one -data unit equals one character, so all offsets are valid). Unlike the pattern -string, the subject may contain binary zeroes. -.P -A non-zero starting offset is useful when searching for another match in the -same subject by calling \fBpcre_exec()\fP again after a previous success. -Setting \fIstartoffset\fP differs from just passing over a shortened string and -setting PCRE_NOTBOL in the case of a pattern that begins with any kind of -lookbehind. For example, consider the pattern -.sp - \eBiss\eB -.sp -which finds occurrences of "iss" in the middle of words. (\eB matches only if -the current position in the subject is not a word boundary.) When applied to -the string "Mississipi" the first call to \fBpcre_exec()\fP finds the first -occurrence. If \fBpcre_exec()\fP is called again with just the remainder of the -subject, namely "issipi", it does not match, because \eB is always false at the -start of the subject, which is deemed to be a word boundary. However, if -\fBpcre_exec()\fP is passed the entire string again, but with \fIstartoffset\fP -set to 4, it finds the second occurrence of "iss" because it is able to look -behind the starting point to discover that it is preceded by a letter. -.P -Finding all the matches in a subject is tricky when the pattern can match an -empty string. It is possible to emulate Perl's /g behaviour by first trying the -match again at the same offset, with the PCRE_NOTEMPTY_ATSTART and -PCRE_ANCHORED options, and then if that fails, advancing the starting offset -and trying an ordinary match again. There is some code that demonstrates how to -do this in the -.\" HREF -\fBpcredemo\fP -.\" -sample program. In the most general case, you have to check to see if the -newline convention recognizes CRLF as a newline, and if so, and the current -character is CR followed by LF, advance the starting offset by two characters -instead of one. -.P -If a non-zero starting offset is passed when the pattern is anchored, one -attempt to match at the given offset is made. This can only succeed if the -pattern does not require the match to be at the start of the subject. -. -. -.SS "How \fBpcre_exec()\fP returns captured substrings" -.rs -.sp -In general, a pattern matches a certain portion of the subject, and in -addition, further substrings from the subject may be picked out by parts of the -pattern. Following the usage in Jeffrey Friedl's book, this is called -"capturing" in what follows, and the phrase "capturing subpattern" is used for -a fragment of a pattern that picks out a substring. PCRE supports several other -kinds of parenthesized subpattern that do not cause substrings to be captured. -.P -Captured substrings are returned to the caller via a vector of integers whose -address is passed in \fIovector\fP. The number of elements in the vector is -passed in \fIovecsize\fP, which must be a non-negative number. \fBNote\fP: this -argument is NOT the size of \fIovector\fP in bytes. -.P -The first two-thirds of the vector is used to pass back captured substrings, -each substring using a pair of integers. The remaining third of the vector is -used as workspace by \fBpcre_exec()\fP while matching capturing subpatterns, -and is not available for passing back information. The number passed in -\fIovecsize\fP should always be a multiple of three. If it is not, it is -rounded down. -.P -When a match is successful, information about captured substrings is returned -in pairs of integers, starting at the beginning of \fIovector\fP, and -continuing up to two-thirds of its length at the most. The first element of -each pair is set to the offset of the first character in a substring, and the -second is set to the offset of the first character after the end of a -substring. These values are always data unit offsets, even in UTF mode. They -are byte offsets in the 8-bit library, 16-bit data item offsets in the 16-bit -library, and 32-bit data item offsets in the 32-bit library. \fBNote\fP: they -are not character counts. -.P -The first pair of integers, \fIovector[0]\fP and \fIovector[1]\fP, identify the -portion of the subject string matched by the entire pattern. The next pair is -used for the first capturing subpattern, and so on. The value returned by -\fBpcre_exec()\fP is one more than the highest numbered pair that has been set. -For example, if two substrings have been captured, the returned value is 3. If -there are no capturing subpatterns, the return value from a successful match is -1, indicating that just the first pair of offsets has been set. -.P -If a capturing subpattern is matched repeatedly, it is the last portion of the -string that it matched that is returned. -.P -If the vector is too small to hold all the captured substring offsets, it is -used as far as possible (up to two-thirds of its length), and the function -returns a value of zero. If neither the actual string matched nor any captured -substrings are of interest, \fBpcre_exec()\fP may be called with \fIovector\fP -passed as NULL and \fIovecsize\fP as zero. However, if the pattern contains -back references and the \fIovector\fP is not big enough to remember the related -substrings, PCRE has to get additional memory for use during matching. Thus it -is usually advisable to supply an \fIovector\fP of reasonable size. -.P -There are some cases where zero is returned (indicating vector overflow) when -in fact the vector is exactly the right size for the final match. For example, -consider the pattern -.sp - (a)(?:(b)c|bd) -.sp -If a vector of 6 elements (allowing for only 1 captured substring) is given -with subject string "abd", \fBpcre_exec()\fP will try to set the second -captured string, thereby recording a vector overflow, before failing to match -"c" and backing up to try the second alternative. The zero return, however, -does correctly indicate that the maximum number of slots (namely 2) have been -filled. In similar cases where there is temporary overflow, but the final -number of used slots is actually less than the maximum, a non-zero value is -returned. -.P -The \fBpcre_fullinfo()\fP function can be used to find out how many capturing -subpatterns there are in a compiled pattern. The smallest size for -\fIovector\fP that will allow for \fIn\fP captured substrings, in addition to -the offsets of the substring matched by the whole pattern, is (\fIn\fP+1)*3. -.P -It is possible for capturing subpattern number \fIn+1\fP to match some part of -the subject when subpattern \fIn\fP has not been used at all. For example, if -the string "abc" is matched against the pattern (a|(z))(bc) the return from the -function is 4, and subpatterns 1 and 3 are matched, but 2 is not. When this -happens, both values in the offset pairs corresponding to unused subpatterns -are set to -1. -.P -Offset values that correspond to unused subpatterns at the end of the -expression are also set to -1. For example, if the string "abc" is matched -against the pattern (abc)(x(yz)?)? subpatterns 2 and 3 are not matched. The -return from the function is 2, because the highest used capturing subpattern -number is 1, and the offsets for for the second and third capturing subpatterns -(assuming the vector is large enough, of course) are set to -1. -.P -\fBNote\fP: Elements in the first two-thirds of \fIovector\fP that do not -correspond to capturing parentheses in the pattern are never changed. That is, -if a pattern contains \fIn\fP capturing parentheses, no more than -\fIovector[0]\fP to \fIovector[2n+1]\fP are set by \fBpcre_exec()\fP. The other -elements (in the first two-thirds) retain whatever values they previously had. -.P -Some convenience functions are provided for extracting the captured substrings -as separate strings. These are described below. -. -. -.\" HTML <a name="errorlist"></a> -.SS "Error return values from \fBpcre_exec()\fP" -.rs -.sp -If \fBpcre_exec()\fP fails, it returns a negative number. The following are -defined in the header file: -.sp - PCRE_ERROR_NOMATCH (-1) -.sp -The subject string did not match the pattern. -.sp - PCRE_ERROR_NULL (-2) -.sp -Either \fIcode\fP or \fIsubject\fP was passed as NULL, or \fIovector\fP was -NULL and \fIovecsize\fP was not zero. -.sp - PCRE_ERROR_BADOPTION (-3) -.sp -An unrecognized bit was set in the \fIoptions\fP argument. -.sp - PCRE_ERROR_BADMAGIC (-4) -.sp -PCRE stores a 4-byte "magic number" at the start of the compiled code, to catch -the case when it is passed a junk pointer and to detect when a pattern that was -compiled in an environment of one endianness is run in an environment with the -other endianness. This is the error that PCRE gives when the magic number is -not present. -.sp - PCRE_ERROR_UNKNOWN_OPCODE (-5) -.sp -While running the pattern match, an unknown item was encountered in the -compiled pattern. This error could be caused by a bug in PCRE or by overwriting -of the compiled pattern. -.sp - PCRE_ERROR_NOMEMORY (-6) -.sp -If a pattern contains back references, but the \fIovector\fP that is passed to -\fBpcre_exec()\fP is not big enough to remember the referenced substrings, PCRE -gets a block of memory at the start of matching to use for this purpose. If the -call via \fBpcre_malloc()\fP fails, this error is given. The memory is -automatically freed at the end of matching. -.P -This error is also given if \fBpcre_stack_malloc()\fP fails in -\fBpcre_exec()\fP. This can happen only when PCRE has been compiled with -\fB--disable-stack-for-recursion\fP. -.sp - PCRE_ERROR_NOSUBSTRING (-7) -.sp -This error is used by the \fBpcre_copy_substring()\fP, -\fBpcre_get_substring()\fP, and \fBpcre_get_substring_list()\fP functions (see -below). It is never returned by \fBpcre_exec()\fP. -.sp - PCRE_ERROR_MATCHLIMIT (-8) -.sp -The backtracking limit, as specified by the \fImatch_limit\fP field in a -\fBpcre_extra\fP structure (or defaulted) was reached. See the description -above. -.sp - PCRE_ERROR_CALLOUT (-9) -.sp -This error is never generated by \fBpcre_exec()\fP itself. It is provided for -use by callout functions that want to yield a distinctive error code. See the -.\" HREF -\fBpcrecallout\fP -.\" -documentation for details. -.sp - PCRE_ERROR_BADUTF8 (-10) -.sp -A string that contains an invalid UTF-8 byte sequence was passed as a subject, -and the PCRE_NO_UTF8_CHECK option was not set. If the size of the output vector -(\fIovecsize\fP) is at least 2, the byte offset to the start of the the invalid -UTF-8 character is placed in the first element, and a reason code is placed in -the second element. The reason codes are listed in the -.\" HTML <a href="#badutf8reasons"> -.\" </a> -following section. -.\" -For backward compatibility, if PCRE_PARTIAL_HARD is set and the problem is a -truncated UTF-8 character at the end of the subject (reason codes 1 to 5), -PCRE_ERROR_SHORTUTF8 is returned instead of PCRE_ERROR_BADUTF8. -.sp - PCRE_ERROR_BADUTF8_OFFSET (-11) -.sp -The UTF-8 byte sequence that was passed as a subject was checked and found to -be valid (the PCRE_NO_UTF8_CHECK option was not set), but the value of -\fIstartoffset\fP did not point to the beginning of a UTF-8 character or the -end of the subject. -.sp - PCRE_ERROR_PARTIAL (-12) -.sp -The subject string did not match, but it did match partially. See the -.\" HREF -\fBpcrepartial\fP -.\" -documentation for details of partial matching. -.sp - PCRE_ERROR_BADPARTIAL (-13) -.sp -This code is no longer in use. It was formerly returned when the PCRE_PARTIAL -option was used with a compiled pattern containing items that were not -supported for partial matching. From release 8.00 onwards, there are no -restrictions on partial matching. -.sp - PCRE_ERROR_INTERNAL (-14) -.sp -An unexpected internal error has occurred. This error could be caused by a bug -in PCRE or by overwriting of the compiled pattern. -.sp - PCRE_ERROR_BADCOUNT (-15) -.sp -This error is given if the value of the \fIovecsize\fP argument is negative. -.sp - PCRE_ERROR_RECURSIONLIMIT (-21) -.sp -The internal recursion limit, as specified by the \fImatch_limit_recursion\fP -field in a \fBpcre_extra\fP structure (or defaulted) was reached. See the -description above. -.sp - PCRE_ERROR_BADNEWLINE (-23) -.sp -An invalid combination of PCRE_NEWLINE_\fIxxx\fP options was given. -.sp - PCRE_ERROR_BADOFFSET (-24) -.sp -The value of \fIstartoffset\fP was negative or greater than the length of the -subject, that is, the value in \fIlength\fP. -.sp - PCRE_ERROR_SHORTUTF8 (-25) -.sp -This error is returned instead of PCRE_ERROR_BADUTF8 when the subject string -ends with a truncated UTF-8 character and the PCRE_PARTIAL_HARD option is set. -Information about the failure is returned as for PCRE_ERROR_BADUTF8. It is in -fact sufficient to detect this case, but this special error code for -PCRE_PARTIAL_HARD precedes the implementation of returned information; it is -retained for backwards compatibility. -.sp - PCRE_ERROR_RECURSELOOP (-26) -.sp -This error is returned when \fBpcre_exec()\fP detects a recursion loop within -the pattern. Specifically, it means that either the whole pattern or a -subpattern has been called recursively for the second time at the same position -in the subject string. Some simple patterns that might do this are detected and -faulted at compile time, but more complicated cases, in particular mutual -recursions between two different subpatterns, cannot be detected until run -time. -.sp - PCRE_ERROR_JIT_STACKLIMIT (-27) -.sp -This error is returned when a pattern that was successfully studied using a -JIT compile option is being matched, but the memory available for the -just-in-time processing stack is not large enough. See the -.\" HREF -\fBpcrejit\fP -.\" -documentation for more details. -.sp - PCRE_ERROR_BADMODE (-28) -.sp -This error is given if a pattern that was compiled by the 8-bit library is -passed to a 16-bit or 32-bit library function, or vice versa. -.sp - PCRE_ERROR_BADENDIANNESS (-29) -.sp -This error is given if a pattern that was compiled and saved is reloaded on a -host with different endianness. The utility function -\fBpcre_pattern_to_host_byte_order()\fP can be used to convert such a pattern -so that it runs on the new host. -.sp - PCRE_ERROR_JIT_BADOPTION -.sp -This error is returned when a pattern that was successfully studied using a JIT -compile option is being matched, but the matching mode (partial or complete -match) does not correspond to any JIT compilation mode. When the JIT fast path -function is used, this error may be also given for invalid options. See the -.\" HREF -\fBpcrejit\fP -.\" -documentation for more details. -.sp - PCRE_ERROR_BADLENGTH (-32) -.sp -This error is given if \fBpcre_exec()\fP is called with a negative value for -the \fIlength\fP argument. -.P -Error numbers -16 to -20, -22, and 30 are not used by \fBpcre_exec()\fP. -. -. -.\" HTML <a name="badutf8reasons"></a> -.SS "Reason codes for invalid UTF-8 strings" -.rs -.sp -This section applies only to the 8-bit library. The corresponding information -for the 16-bit and 32-bit libraries is given in the -.\" HREF -\fBpcre16\fP -.\" -and -.\" HREF -\fBpcre32\fP -.\" -pages. -.P -When \fBpcre_exec()\fP returns either PCRE_ERROR_BADUTF8 or -PCRE_ERROR_SHORTUTF8, and the size of the output vector (\fIovecsize\fP) is at -least 2, the offset of the start of the invalid UTF-8 character is placed in -the first output vector element (\fIovector[0]\fP) and a reason code is placed -in the second element (\fIovector[1]\fP). The reason codes are given names in -the \fBpcre.h\fP header file: -.sp - PCRE_UTF8_ERR1 - PCRE_UTF8_ERR2 - PCRE_UTF8_ERR3 - PCRE_UTF8_ERR4 - PCRE_UTF8_ERR5 -.sp -The string ends with a truncated UTF-8 character; the code specifies how many -bytes are missing (1 to 5). Although RFC 3629 restricts UTF-8 characters to be -no longer than 4 bytes, the encoding scheme (originally defined by RFC 2279) -allows for up to 6 bytes, and this is checked first; hence the possibility of -4 or 5 missing bytes. -.sp - PCRE_UTF8_ERR6 - PCRE_UTF8_ERR7 - PCRE_UTF8_ERR8 - PCRE_UTF8_ERR9 - PCRE_UTF8_ERR10 -.sp -The two most significant bits of the 2nd, 3rd, 4th, 5th, or 6th byte of the -character do not have the binary value 0b10 (that is, either the most -significant bit is 0, or the next bit is 1). -.sp - PCRE_UTF8_ERR11 - PCRE_UTF8_ERR12 -.sp -A character that is valid by the RFC 2279 rules is either 5 or 6 bytes long; -these code points are excluded by RFC 3629. -.sp - PCRE_UTF8_ERR13 -.sp -A 4-byte character has a value greater than 0x10fff; these code points are -excluded by RFC 3629. -.sp - PCRE_UTF8_ERR14 -.sp -A 3-byte character has a value in the range 0xd800 to 0xdfff; this range of -code points are reserved by RFC 3629 for use with UTF-16, and so are excluded -from UTF-8. -.sp - PCRE_UTF8_ERR15 - PCRE_UTF8_ERR16 - PCRE_UTF8_ERR17 - PCRE_UTF8_ERR18 - PCRE_UTF8_ERR19 -.sp -A 2-, 3-, 4-, 5-, or 6-byte character is "overlong", that is, it codes for a -value that can be represented by fewer bytes, which is invalid. For example, -the two bytes 0xc0, 0xae give the value 0x2e, whose correct coding uses just -one byte. -.sp - PCRE_UTF8_ERR20 -.sp -The two most significant bits of the first byte of a character have the binary -value 0b10 (that is, the most significant bit is 1 and the second is 0). Such a -byte can only validly occur as the second or subsequent byte of a multi-byte -character. -.sp - PCRE_UTF8_ERR21 -.sp -The first byte of a character has the value 0xfe or 0xff. These values can -never occur in a valid UTF-8 string. -.sp - PCRE_UTF8_ERR22 -.sp -This error code was formerly used when the presence of a so-called -"non-character" caused an error. Unicode corrigendum #9 makes it clear that -such characters should not cause a string to be rejected, and so this code is -no longer in use and is never returned. -. -. -.SH "EXTRACTING CAPTURED SUBSTRINGS BY NUMBER" -.rs -.sp -.nf -.B int pcre_copy_substring(const char *\fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP, char *\fIbuffer\fP," -.B " int \fIbuffersize\fP);" -.sp -.B int pcre_get_substring(const char *\fIsubject\fP, int *\fIovector\fP, -.B " int \fIstringcount\fP, int \fIstringnumber\fP," -.B " const char **\fIstringptr\fP);" -.sp -.B int pcre_get_substring_list(const char *\fIsubject\fP, -.B " int *\fIovector\fP, int \fIstringcount\fP, const char ***\fIlistptr\fP);" -.fi -.PP -Captured substrings can be accessed directly by using the offsets returned by -\fBpcre_exec()\fP in \fIovector\fP. For convenience, the functions -\fBpcre_copy_substring()\fP, \fBpcre_get_substring()\fP, and -\fBpcre_get_substring_list()\fP are provided for extracting captured substrings -as new, separate, zero-terminated strings. These functions identify substrings -by number. The next section describes functions for extracting named -substrings. -.P -A substring that contains a binary zero is correctly extracted and has a -further zero added on the end, but the result is not, of course, a C string. -However, you can process such a string by referring to the length that is -returned by \fBpcre_copy_substring()\fP and \fBpcre_get_substring()\fP. -Unfortunately, the interface to \fBpcre_get_substring_list()\fP is not adequate -for handling strings containing binary zeros, because the end of the final -string is not independently indicated. -.P -The first three arguments are the same for all three of these functions: -\fIsubject\fP is the subject string that has just been successfully matched, -\fIovector\fP is a pointer to the vector of integer offsets that was passed to -\fBpcre_exec()\fP, and \fIstringcount\fP is the number of substrings that were -captured by the match, including the substring that matched the entire regular -expression. This is the value returned by \fBpcre_exec()\fP if it is greater -than zero. If \fBpcre_exec()\fP returned zero, indicating that it ran out of -space in \fIovector\fP, the value passed as \fIstringcount\fP should be the -number of elements in the vector divided by three. -.P -The functions \fBpcre_copy_substring()\fP and \fBpcre_get_substring()\fP -extract a single substring, whose number is given as \fIstringnumber\fP. A -value of zero extracts the substring that matched the entire pattern, whereas -higher values extract the captured substrings. For \fBpcre_copy_substring()\fP, -the string is placed in \fIbuffer\fP, whose length is given by -\fIbuffersize\fP, while for \fBpcre_get_substring()\fP a new block of memory is -obtained via \fBpcre_malloc\fP, and its address is returned via -\fIstringptr\fP. The yield of the function is the length of the string, not -including the terminating zero, or one of these error codes: -.sp - PCRE_ERROR_NOMEMORY (-6) -.sp -The buffer was too small for \fBpcre_copy_substring()\fP, or the attempt to get -memory failed for \fBpcre_get_substring()\fP. -.sp - PCRE_ERROR_NOSUBSTRING (-7) -.sp -There is no substring whose number is \fIstringnumber\fP. -.P -The \fBpcre_get_substring_list()\fP function extracts all available substrings -and builds a list of pointers to them. All this is done in a single block of -memory that is obtained via \fBpcre_malloc\fP. The address of the memory block -is returned via \fIlistptr\fP, which is also the start of the list of string -pointers. The end of the list is marked by a NULL pointer. The yield of the -function is zero if all went well, or the error code -.sp - PCRE_ERROR_NOMEMORY (-6) -.sp -if the attempt to get the memory block failed. -.P -When any of these functions encounter a substring that is unset, which can -happen when capturing subpattern number \fIn+1\fP matches some part of the -subject, but subpattern \fIn\fP has not been used at all, they return an empty -string. This can be distinguished from a genuine zero-length substring by -inspecting the appropriate offset in \fIovector\fP, which is negative for unset -substrings. -.P -The two convenience functions \fBpcre_free_substring()\fP and -\fBpcre_free_substring_list()\fP can be used to free the memory returned by -a previous call of \fBpcre_get_substring()\fP or -\fBpcre_get_substring_list()\fP, respectively. They do nothing more than call -the function pointed to by \fBpcre_free\fP, which of course could be called -directly from a C program. However, PCRE is used in some situations where it is -linked via a special interface to another programming language that cannot use -\fBpcre_free\fP directly; it is for these cases that the functions are -provided. -. -. -.SH "EXTRACTING CAPTURED SUBSTRINGS BY NAME" -.rs -.sp -.nf -.B int pcre_get_stringnumber(const pcre *\fIcode\fP, -.B " const char *\fIname\fP);" -.sp -.B int pcre_copy_named_substring(const pcre *\fIcode\fP, -.B " const char *\fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, const char *\fIstringname\fP," -.B " char *\fIbuffer\fP, int \fIbuffersize\fP);" -.sp -.B int pcre_get_named_substring(const pcre *\fIcode\fP, -.B " const char *\fIsubject\fP, int *\fIovector\fP," -.B " int \fIstringcount\fP, const char *\fIstringname\fP," -.B " const char **\fIstringptr\fP);" -.fi -.PP -To extract a substring by name, you first have to find associated number. -For example, for this pattern -.sp - (a+)b(?<xxx>\ed+)... -.sp -the number of the subpattern called "xxx" is 2. If the name is known to be -unique (PCRE_DUPNAMES was not set), you can find the number from the name by -calling \fBpcre_get_stringnumber()\fP. The first argument is the compiled -pattern, and the second is the name. The yield of the function is the -subpattern number, or PCRE_ERROR_NOSUBSTRING (-7) if there is no subpattern of -that name. -.P -Given the number, you can extract the substring directly, or use one of the -functions described in the previous section. For convenience, there are also -two functions that do the whole job. -.P -Most of the arguments of \fBpcre_copy_named_substring()\fP and -\fBpcre_get_named_substring()\fP are the same as those for the similarly named -functions that extract by number. As these are described in the previous -section, they are not re-described here. There are just two differences: -.P -First, instead of a substring number, a substring name is given. Second, there -is an extra argument, given at the start, which is a pointer to the compiled -pattern. This is needed in order to gain access to the name-to-number -translation table. -.P -These functions call \fBpcre_get_stringnumber()\fP, and if it succeeds, they -then call \fBpcre_copy_substring()\fP or \fBpcre_get_substring()\fP, as -appropriate. \fBNOTE:\fP If PCRE_DUPNAMES is set and there are duplicate names, -the behaviour may not be what you want (see the next section). -.P -\fBWarning:\fP If the pattern uses the (?| feature to set up multiple -subpatterns with the same number, as described in the -.\" HTML <a href="pcrepattern.html#dupsubpatternnumber"> -.\" </a> -section on duplicate subpattern numbers -.\" -in the -.\" HREF -\fBpcrepattern\fP -.\" -page, you cannot use names to distinguish the different subpatterns, because -names are not included in the compiled code. The matching process uses only -numbers. For this reason, the use of different names for subpatterns of the -same number causes an error at compile time. -. -. -.SH "DUPLICATE SUBPATTERN NAMES" -.rs -.sp -.nf -.B int pcre_get_stringtable_entries(const pcre *\fIcode\fP, -.B " const char *\fIname\fP, char **\fIfirst\fP, char **\fIlast\fP);" -.fi -.PP -When a pattern is compiled with the PCRE_DUPNAMES option, names for subpatterns -are not required to be unique. (Duplicate names are always allowed for -subpatterns with the same number, created by using the (?| feature. Indeed, if -such subpatterns are named, they are required to use the same names.) -.P -Normally, patterns with duplicate names are such that in any one match, only -one of the named subpatterns participates. An example is shown in the -.\" HREF -\fBpcrepattern\fP -.\" -documentation. -.P -When duplicates are present, \fBpcre_copy_named_substring()\fP and -\fBpcre_get_named_substring()\fP return the first substring corresponding to -the given name that is set. If none are set, PCRE_ERROR_NOSUBSTRING (-7) is -returned; no data is returned. The \fBpcre_get_stringnumber()\fP function -returns one of the numbers that are associated with the name, but it is not -defined which it is. -.P -If you want to get full details of all captured substrings for a given name, -you must use the \fBpcre_get_stringtable_entries()\fP function. The first -argument is the compiled pattern, and the second is the name. The third and -fourth are pointers to variables which are updated by the function. After it -has run, they point to the first and last entries in the name-to-number table -for the given name. The function itself returns the length of each entry, or -PCRE_ERROR_NOSUBSTRING (-7) if there are none. The format of the table is -described above in the section entitled \fIInformation about a pattern\fP -.\" HTML <a href="#infoaboutpattern"> -.\" </a> -above. -.\" -Given all the relevant entries for the name, you can extract each of their -numbers, and hence the captured data, if any. -. -. -.SH "FINDING ALL POSSIBLE MATCHES" -.rs -.sp -The traditional matching function uses a similar algorithm to Perl, which stops -when it finds the first match, starting at a given point in the subject. If you -want to find all possible matches, or the longest possible match, consider -using the alternative matching function (see below) instead. If you cannot use -the alternative function, but still need to find all possible matches, you -can kludge it up by making use of the callout facility, which is described in -the -.\" HREF -\fBpcrecallout\fP -.\" -documentation. -.P -What you have to do is to insert a callout right at the end of the pattern. -When your callout function is called, extract and save the current matched -substring. Then return 1, which forces \fBpcre_exec()\fP to backtrack and try -other alternatives. Ultimately, when it runs out of matches, \fBpcre_exec()\fP -will yield PCRE_ERROR_NOMATCH. -. -. -.SH "OBTAINING AN ESTIMATE OF STACK USAGE" -.rs -.sp -Matching certain patterns using \fBpcre_exec()\fP can use a lot of process -stack, which in certain environments can be rather limited in size. Some users -find it helpful to have an estimate of the amount of stack that is used by -\fBpcre_exec()\fP, to help them set recursion limits, as described in the -.\" HREF -\fBpcrestack\fP -.\" -documentation. The estimate that is output by \fBpcretest\fP when called with -the \fB-m\fP and \fB-C\fP options is obtained by calling \fBpcre_exec\fP with -the values NULL, NULL, NULL, -999, and -999 for its first five arguments. -.P -Normally, if its first argument is NULL, \fBpcre_exec()\fP immediately returns -the negative error code PCRE_ERROR_NULL, but with this special combination of -arguments, it returns instead a negative number whose absolute value is the -approximate stack frame size in bytes. (A negative number is used so that it is -clear that no match has happened.) The value is approximate because in some -cases, recursive calls to \fBpcre_exec()\fP occur when there are one or two -additional variables on the stack. -.P -If PCRE has been compiled to use the heap instead of the stack for recursion, -the value returned is the size of each block that is obtained from the heap. -. -. -.\" HTML <a name="dfamatch"></a> -.SH "MATCHING A PATTERN: THE ALTERNATIVE FUNCTION" -.rs -.sp -.nf -.B int pcre_dfa_exec(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," -.B " const char *\fIsubject\fP, int \fIlength\fP, int \fIstartoffset\fP," -.B " int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP," -.B " int *\fIworkspace\fP, int \fIwscount\fP);" -.fi -.P -The function \fBpcre_dfa_exec()\fP is called to match a subject string against -a compiled pattern, using a matching algorithm that scans the subject string -just once, and does not backtrack. This has different characteristics to the -normal algorithm, and is not compatible with Perl. Some of the features of PCRE -patterns are not supported. Nevertheless, there are times when this kind of -matching can be useful. For a discussion of the two matching algorithms, and a -list of features that \fBpcre_dfa_exec()\fP does not support, see the -.\" HREF -\fBpcrematching\fP -.\" -documentation. -.P -The arguments for the \fBpcre_dfa_exec()\fP function are the same as for -\fBpcre_exec()\fP, plus two extras. The \fIovector\fP argument is used in a -different way, and this is described below. The other common arguments are used -in the same way as for \fBpcre_exec()\fP, so their description is not repeated -here. -.P -The two additional arguments provide workspace for the function. The workspace -vector should contain at least 20 elements. It is used for keeping track of -multiple paths through the pattern tree. More workspace will be needed for -patterns and subjects where there are a lot of potential matches. -.P -Here is an example of a simple call to \fBpcre_dfa_exec()\fP: -.sp - int rc; - int ovector[10]; - int wspace[20]; - rc = pcre_dfa_exec( - re, /* result of pcre_compile() */ - NULL, /* we didn't study the pattern */ - "some string", /* the subject string */ - 11, /* the length of the subject string */ - 0, /* start at offset 0 in the subject */ - 0, /* default options */ - ovector, /* vector of integers for substring information */ - 10, /* number of elements (NOT size in bytes) */ - wspace, /* working space vector */ - 20); /* number of elements (NOT size in bytes) */ -. -.SS "Option bits for \fBpcre_dfa_exec()\fP" -.rs -.sp -The unused bits of the \fIoptions\fP argument for \fBpcre_dfa_exec()\fP must be -zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_\fIxxx\fP, -PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, -PCRE_NO_UTF8_CHECK, PCRE_BSR_ANYCRLF, PCRE_BSR_UNICODE, PCRE_NO_START_OPTIMIZE, -PCRE_PARTIAL_HARD, PCRE_PARTIAL_SOFT, PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. -All but the last four of these are exactly the same as for \fBpcre_exec()\fP, -so their description is not repeated here. -.sp - PCRE_PARTIAL_HARD - PCRE_PARTIAL_SOFT -.sp -These have the same general effect as they do for \fBpcre_exec()\fP, but the -details are slightly different. When PCRE_PARTIAL_HARD is set for -\fBpcre_dfa_exec()\fP, it returns PCRE_ERROR_PARTIAL if the end of the subject -is reached and there is still at least one matching possibility that requires -additional characters. This happens even if some complete matches have also -been found. When PCRE_PARTIAL_SOFT is set, the return code PCRE_ERROR_NOMATCH -is converted into PCRE_ERROR_PARTIAL if the end of the subject is reached, -there have been no complete matches, but there is still at least one matching -possibility. The portion of the string that was inspected when the longest -partial match was found is set as the first matching string in both cases. -There is a more detailed discussion of partial and multi-segment matching, with -examples, in the -.\" HREF -\fBpcrepartial\fP -.\" -documentation. -.sp - PCRE_DFA_SHORTEST -.sp -Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to stop as -soon as it has found one match. Because of the way the alternative algorithm -works, this is necessarily the shortest possible match at the first possible -matching point in the subject string. -.sp - PCRE_DFA_RESTART -.sp -When \fBpcre_dfa_exec()\fP returns a partial match, it is possible to call it -again, with additional subject characters, and have it continue with the same -match. The PCRE_DFA_RESTART option requests this action; when it is set, the -\fIworkspace\fP and \fIwscount\fP options must reference the same vector as -before because data about the match so far is left in them after a partial -match. There is more discussion of this facility in the -.\" HREF -\fBpcrepartial\fP -.\" -documentation. -. -. -.SS "Successful returns from \fBpcre_dfa_exec()\fP" -.rs -.sp -When \fBpcre_dfa_exec()\fP succeeds, it may have matched more than one -substring in the subject. Note, however, that all the matches from one run of -the function start at the same point in the subject. The shorter matches are -all initial substrings of the longer matches. For example, if the pattern -.sp - <.*> -.sp -is matched against the string -.sp - This is <something> <something else> <something further> no more -.sp -the three matched strings are -.sp - <something> - <something> <something else> - <something> <something else> <something further> -.sp -On success, the yield of the function is a number greater than zero, which is -the number of matched substrings. The substrings themselves are returned in -\fIovector\fP. Each string uses two elements; the first is the offset to the -start, and the second is the offset to the end. In fact, all the strings have -the same start offset. (Space could have been saved by giving this only once, -but it was decided to retain some compatibility with the way \fBpcre_exec()\fP -returns data, even though the meaning of the strings is different.) -.P -The strings are returned in reverse order of length; that is, the longest -matching string is given first. If there were too many matches to fit into -\fIovector\fP, the yield of the function is zero, and the vector is filled with -the longest matches. Unlike \fBpcre_exec()\fP, \fBpcre_dfa_exec()\fP can use -the entire \fIovector\fP for returning matched strings. -.P -NOTE: PCRE's "auto-possessification" optimization usually applies to character -repeats at the end of a pattern (as well as internally). For example, the -pattern "a\ed+" is compiled as if it were "a\ed++" because there is no point -even considering the possibility of backtracking into the repeated digits. For -DFA matching, this means that only one possible match is found. If you really -do want multiple matches in such cases, either use an ungreedy repeat -("a\ed+?") or set the PCRE_NO_AUTO_POSSESS option when compiling. -. -. -.SS "Error returns from \fBpcre_dfa_exec()\fP" -.rs -.sp -The \fBpcre_dfa_exec()\fP function returns a negative number when it fails. -Many of the errors are the same as for \fBpcre_exec()\fP, and these are -described -.\" HTML <a href="#errorlist"> -.\" </a> -above. -.\" -There are in addition the following errors that are specific to -\fBpcre_dfa_exec()\fP: -.sp - PCRE_ERROR_DFA_UITEM (-16) -.sp -This return is given if \fBpcre_dfa_exec()\fP encounters an item in the pattern -that it does not support, for instance, the use of \eC or a back reference. -.sp - PCRE_ERROR_DFA_UCOND (-17) -.sp -This return is given if \fBpcre_dfa_exec()\fP encounters a condition item that -uses a back reference for the condition, or a test for recursion in a specific -group. These are not supported. -.sp - PCRE_ERROR_DFA_UMLIMIT (-18) -.sp -This return is given if \fBpcre_dfa_exec()\fP is called with an \fIextra\fP -block that contains a setting of the \fImatch_limit\fP or -\fImatch_limit_recursion\fP fields. This is not supported (these fields are -meaningless for DFA matching). -.sp - PCRE_ERROR_DFA_WSSIZE (-19) -.sp -This return is given if \fBpcre_dfa_exec()\fP runs out of space in the -\fIworkspace\fP vector. -.sp - PCRE_ERROR_DFA_RECURSE (-20) -.sp -When a recursive subpattern is processed, the matching function calls itself -recursively, using private vectors for \fIovector\fP and \fIworkspace\fP. This -error is given if the output vector is not large enough. This should be -extremely rare, as a vector of size 1000 is used. -.sp - PCRE_ERROR_DFA_BADRESTART (-30) -.sp -When \fBpcre_dfa_exec()\fP is called with the \fBPCRE_DFA_RESTART\fP option, -some plausibility checks are made on the contents of the workspace, which -should contain data about the previous partial match. If any of these checks -fail, this error is given. -. -. -.SH "SEE ALSO" -.rs -.sp -\fBpcre16\fP(3), \fBpcre32\fP(3), \fBpcrebuild\fP(3), \fBpcrecallout\fP(3), -\fBpcrecpp(3)\fP(3), \fBpcrematching\fP(3), \fBpcrepartial\fP(3), -\fBpcreposix\fP(3), \fBpcreprecompile\fP(3), \fBpcresample\fP(3), -\fBpcrestack\fP(3). -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 09 February 2014 -Copyright (c) 1997-2014 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcrebuild.3 b/plugins/Pcre16/docs/doc/pcrebuild.3 deleted file mode 100644 index 403f2ae32f..0000000000 --- a/plugins/Pcre16/docs/doc/pcrebuild.3 +++ /dev/null @@ -1,550 +0,0 @@ -.TH PCREBUILD 3 "12 May 2013" "PCRE 8.33" -.SH NAME -PCRE - Perl-compatible regular expressions -. -. -.SH "BUILDING PCRE" -.rs -.sp -PCRE is distributed with a \fBconfigure\fP script that can be used to build the -library in Unix-like environments using the applications known as Autotools. -Also in the distribution are files to support building using \fBCMake\fP -instead of \fBconfigure\fP. The text file -.\" HTML <a href="README.txt"> -.\" </a> -\fBREADME\fP -.\" -contains general information about building with Autotools (some of which is -repeated below), and also has some comments about building on various operating -systems. There is a lot more information about building PCRE without using -Autotools (including information about using \fBCMake\fP and building "by -hand") in the text file called -.\" HTML <a href="NON-AUTOTOOLS-BUILD.txt"> -.\" </a> -\fBNON-AUTOTOOLS-BUILD\fP. -.\" -You should consult this file as well as the -.\" HTML <a href="README.txt"> -.\" </a> -\fBREADME\fP -.\" -file if you are building in a non-Unix-like environment. -. -. -.SH "PCRE BUILD-TIME OPTIONS" -.rs -.sp -The rest of this document describes the optional features of PCRE that can be -selected when the library is compiled. It assumes use of the \fBconfigure\fP -script, where the optional features are selected or deselected by providing -options to \fBconfigure\fP before running the \fBmake\fP command. However, the -same options can be selected in both Unix-like and non-Unix-like environments -using the GUI facility of \fBcmake-gui\fP if you are using \fBCMake\fP instead -of \fBconfigure\fP to build PCRE. -.P -If you are not using Autotools or \fBCMake\fP, option selection can be done by -editing the \fBconfig.h\fP file, or by passing parameter settings to the -compiler, as described in -.\" HTML <a href="NON-AUTOTOOLS-BUILD.txt"> -.\" </a> -\fBNON-AUTOTOOLS-BUILD\fP. -.\" -.P -The complete list of options for \fBconfigure\fP (which includes the standard -ones such as the selection of the installation directory) can be obtained by -running -.sp - ./configure --help -.sp -The following sections include descriptions of options whose names begin with ---enable or --disable. These settings specify changes to the defaults for the -\fBconfigure\fP command. Because of the way that \fBconfigure\fP works, ---enable and --disable always come in pairs, so the complementary option always -exists as well, but as it specifies the default, it is not described. -. -. -.SH "BUILDING 8-BIT, 16-BIT AND 32-BIT LIBRARIES" -.rs -.sp -By default, a library called \fBlibpcre\fP is built, containing functions that -take string arguments contained in vectors of bytes, either as single-byte -characters, or interpreted as UTF-8 strings. You can also build a separate -library, called \fBlibpcre16\fP, in which strings are contained in vectors of -16-bit data units and interpreted either as single-unit characters or UTF-16 -strings, by adding -.sp - --enable-pcre16 -.sp -to the \fBconfigure\fP command. You can also build yet another separate -library, called \fBlibpcre32\fP, in which strings are contained in vectors of -32-bit data units and interpreted either as single-unit characters or UTF-32 -strings, by adding -.sp - --enable-pcre32 -.sp -to the \fBconfigure\fP command. If you do not want the 8-bit library, add -.sp - --disable-pcre8 -.sp -as well. At least one of the three libraries must be built. Note that the C++ -and POSIX wrappers are for the 8-bit library only, and that \fBpcregrep\fP is -an 8-bit program. None of these are built if you select only the 16-bit or -32-bit libraries. -. -. -.SH "BUILDING SHARED AND STATIC LIBRARIES" -.rs -.sp -The Autotools PCRE building process uses \fBlibtool\fP to build both shared and -static libraries by default. You can suppress one of these by adding one of -.sp - --disable-shared - --disable-static -.sp -to the \fBconfigure\fP command, as required. -. -. -.SH "C++ SUPPORT" -.rs -.sp -By default, if the 8-bit library is being built, the \fBconfigure\fP script -will search for a C++ compiler and C++ header files. If it finds them, it -automatically builds the C++ wrapper library (which supports only 8-bit -strings). You can disable this by adding -.sp - --disable-cpp -.sp -to the \fBconfigure\fP command. -. -. -.SH "UTF-8, UTF-16 AND UTF-32 SUPPORT" -.rs -.sp -To build PCRE with support for UTF Unicode character strings, add -.sp - --enable-utf -.sp -to the \fBconfigure\fP command. This setting applies to all three libraries, -adding support for UTF-8 to the 8-bit library, support for UTF-16 to the 16-bit -library, and support for UTF-32 to the to the 32-bit library. There are no -separate options for enabling UTF-8, UTF-16 and UTF-32 independently because -that would allow ridiculous settings such as requesting UTF-16 support while -building only the 8-bit library. It is not possible to build one library with -UTF support and another without in the same configuration. (For backwards -compatibility, --enable-utf8 is a synonym of --enable-utf.) -.P -Of itself, this setting does not make PCRE treat strings as UTF-8, UTF-16 or -UTF-32. As well as compiling PCRE with this option, you also have have to set -the PCRE_UTF8, PCRE_UTF16 or PCRE_UTF32 option (as appropriate) when you call -one of the pattern compiling functions. -.P -If you set --enable-utf when compiling in an EBCDIC environment, PCRE expects -its input to be either ASCII or UTF-8 (depending on the run-time option). It is -not possible to support both EBCDIC and UTF-8 codes in the same version of the -library. Consequently, --enable-utf and --enable-ebcdic are mutually -exclusive. -. -. -.SH "UNICODE CHARACTER PROPERTY SUPPORT" -.rs -.sp -UTF support allows the libraries to process character codepoints up to 0x10ffff -in the strings that they handle. On its own, however, it does not provide any -facilities for accessing the properties of such characters. If you want to be -able to use the pattern escapes \eP, \ep, and \eX, which refer to Unicode -character properties, you must add -.sp - --enable-unicode-properties -.sp -to the \fBconfigure\fP command. This implies UTF support, even if you have -not explicitly requested it. -.P -Including Unicode property support adds around 30K of tables to the PCRE -library. Only the general category properties such as \fILu\fP and \fINd\fP are -supported. Details are given in the -.\" HREF -\fBpcrepattern\fP -.\" -documentation. -. -. -.SH "JUST-IN-TIME COMPILER SUPPORT" -.rs -.sp -Just-in-time compiler support is included in the build by specifying -.sp - --enable-jit -.sp -This support is available only for certain hardware architectures. If this -option is set for an unsupported architecture, a compile time error occurs. -See the -.\" HREF -\fBpcrejit\fP -.\" -documentation for a discussion of JIT usage. When JIT support is enabled, -pcregrep automatically makes use of it, unless you add -.sp - --disable-pcregrep-jit -.sp -to the "configure" command. -. -. -.SH "CODE VALUE OF NEWLINE" -.rs -.sp -By default, PCRE interprets the linefeed (LF) character as indicating the end -of a line. This is the normal newline character on Unix-like systems. You can -compile PCRE to use carriage return (CR) instead, by adding -.sp - --enable-newline-is-cr -.sp -to the \fBconfigure\fP command. There is also a --enable-newline-is-lf option, -which explicitly specifies linefeed as the newline character. -.sp -Alternatively, you can specify that line endings are to be indicated by the two -character sequence CRLF. If you want this, add -.sp - --enable-newline-is-crlf -.sp -to the \fBconfigure\fP command. There is a fourth option, specified by -.sp - --enable-newline-is-anycrlf -.sp -which causes PCRE to recognize any of the three sequences CR, LF, or CRLF as -indicating a line ending. Finally, a fifth option, specified by -.sp - --enable-newline-is-any -.sp -causes PCRE to recognize any Unicode newline sequence. -.P -Whatever line ending convention is selected when PCRE is built can be -overridden when the library functions are called. At build time it is -conventional to use the standard for your operating system. -. -. -.SH "WHAT \eR MATCHES" -.rs -.sp -By default, the sequence \eR in a pattern matches any Unicode newline sequence, -whatever has been selected as the line ending sequence. If you specify -.sp - --enable-bsr-anycrlf -.sp -the default is changed so that \eR matches only CR, LF, or CRLF. Whatever is -selected when PCRE is built can be overridden when the library functions are -called. -. -. -.SH "POSIX MALLOC USAGE" -.rs -.sp -When the 8-bit library is called through the POSIX interface (see the -.\" HREF -\fBpcreposix\fP -.\" -documentation), additional working storage is required for holding the pointers -to capturing substrings, because PCRE requires three integers per substring, -whereas the POSIX interface provides only two. If the number of expected -substrings is small, the wrapper function uses space on the stack, because this -is faster than using \fBmalloc()\fP for each call. The default threshold above -which the stack is no longer used is 10; it can be changed by adding a setting -such as -.sp - --with-posix-malloc-threshold=20 -.sp -to the \fBconfigure\fP command. -. -. -.SH "HANDLING VERY LARGE PATTERNS" -.rs -.sp -Within a compiled pattern, offset values are used to point from one part to -another (for example, from an opening parenthesis to an alternation -metacharacter). By default, in the 8-bit and 16-bit libraries, two-byte values -are used for these offsets, leading to a maximum size for a compiled pattern of -around 64K. This is sufficient to handle all but the most gigantic patterns. -Nevertheless, some people do want to process truly enormous patterns, so it is -possible to compile PCRE to use three-byte or four-byte offsets by adding a -setting such as -.sp - --with-link-size=3 -.sp -to the \fBconfigure\fP command. The value given must be 2, 3, or 4. For the -16-bit library, a value of 3 is rounded up to 4. In these libraries, using -longer offsets slows down the operation of PCRE because it has to load -additional data when handling them. For the 32-bit library the value is always -4 and cannot be overridden; the value of --with-link-size is ignored. -. -. -.SH "AVOIDING EXCESSIVE STACK USAGE" -.rs -.sp -When matching with the \fBpcre_exec()\fP function, PCRE implements backtracking -by making recursive calls to an internal function called \fBmatch()\fP. In -environments where the size of the stack is limited, this can severely limit -PCRE's operation. (The Unix environment does not usually suffer from this -problem, but it may sometimes be necessary to increase the maximum stack size. -There is a discussion in the -.\" HREF -\fBpcrestack\fP -.\" -documentation.) An alternative approach to recursion that uses memory from the -heap to remember data, instead of using recursive function calls, has been -implemented to work round the problem of limited stack size. If you want to -build a version of PCRE that works this way, add -.sp - --disable-stack-for-recursion -.sp -to the \fBconfigure\fP command. With this configuration, PCRE will use the -\fBpcre_stack_malloc\fP and \fBpcre_stack_free\fP variables to call memory -management functions. By default these point to \fBmalloc()\fP and -\fBfree()\fP, but you can replace the pointers so that your own functions are -used instead. -.P -Separate functions are provided rather than using \fBpcre_malloc\fP and -\fBpcre_free\fP because the usage is very predictable: the block sizes -requested are always the same, and the blocks are always freed in reverse -order. A calling program might be able to implement optimized functions that -perform better than \fBmalloc()\fP and \fBfree()\fP. PCRE runs noticeably more -slowly when built in this way. This option affects only the \fBpcre_exec()\fP -function; it is not relevant for \fBpcre_dfa_exec()\fP. -. -. -.SH "LIMITING PCRE RESOURCE USAGE" -.rs -.sp -Internally, PCRE has a function called \fBmatch()\fP, which it calls repeatedly -(sometimes recursively) when matching a pattern with the \fBpcre_exec()\fP -function. By controlling the maximum number of times this function may be -called during a single matching operation, a limit can be placed on the -resources used by a single call to \fBpcre_exec()\fP. The limit can be changed -at run time, as described in the -.\" HREF -\fBpcreapi\fP -.\" -documentation. The default is 10 million, but this can be changed by adding a -setting such as -.sp - --with-match-limit=500000 -.sp -to the \fBconfigure\fP command. This setting has no effect on the -\fBpcre_dfa_exec()\fP matching function. -.P -In some environments it is desirable to limit the depth of recursive calls of -\fBmatch()\fP more strictly than the total number of calls, in order to -restrict the maximum amount of stack (or heap, if --disable-stack-for-recursion -is specified) that is used. A second limit controls this; it defaults to the -value that is set for --with-match-limit, which imposes no additional -constraints. However, you can set a lower limit by adding, for example, -.sp - --with-match-limit-recursion=10000 -.sp -to the \fBconfigure\fP command. This value can also be overridden at run time. -. -. -.SH "CREATING CHARACTER TABLES AT BUILD TIME" -.rs -.sp -PCRE uses fixed tables for processing characters whose code values are less -than 256. By default, PCRE is built with a set of tables that are distributed -in the file \fIpcre_chartables.c.dist\fP. These tables are for ASCII codes -only. If you add -.sp - --enable-rebuild-chartables -.sp -to the \fBconfigure\fP command, the distributed tables are no longer used. -Instead, a program called \fBdftables\fP is compiled and run. This outputs the -source for new set of tables, created in the default locale of your C run-time -system. (This method of replacing the tables does not work if you are cross -compiling, because \fBdftables\fP is run on the local host. If you need to -create alternative tables when cross compiling, you will have to do so "by -hand".) -. -. -.SH "USING EBCDIC CODE" -.rs -.sp -PCRE assumes by default that it will run in an environment where the character -code is ASCII (or Unicode, which is a superset of ASCII). This is the case for -most computer operating systems. PCRE can, however, be compiled to run in an -EBCDIC environment by adding -.sp - --enable-ebcdic -.sp -to the \fBconfigure\fP command. This setting implies ---enable-rebuild-chartables. You should only use it if you know that you are in -an EBCDIC environment (for example, an IBM mainframe operating system). The ---enable-ebcdic option is incompatible with --enable-utf. -.P -The EBCDIC character that corresponds to an ASCII LF is assumed to have the -value 0x15 by default. However, in some EBCDIC environments, 0x25 is used. In -such an environment you should use -.sp - --enable-ebcdic-nl25 -.sp -as well as, or instead of, --enable-ebcdic. The EBCDIC character for CR has the -same value as in ASCII, namely, 0x0d. Whichever of 0x15 and 0x25 is \fInot\fP -chosen as LF is made to correspond to the Unicode NEL character (which, in -Unicode, is 0x85). -.P -The options that select newline behaviour, such as --enable-newline-is-cr, -and equivalent run-time options, refer to these character values in an EBCDIC -environment. -. -. -.SH "PCREGREP OPTIONS FOR COMPRESSED FILE SUPPORT" -.rs -.sp -By default, \fBpcregrep\fP reads all files as plain text. You can build it so -that it recognizes files whose names end in \fB.gz\fP or \fB.bz2\fP, and reads -them with \fBlibz\fP or \fBlibbz2\fP, respectively, by adding one or both of -.sp - --enable-pcregrep-libz - --enable-pcregrep-libbz2 -.sp -to the \fBconfigure\fP command. These options naturally require that the -relevant libraries are installed on your system. Configuration will fail if -they are not. -. -. -.SH "PCREGREP BUFFER SIZE" -.rs -.sp -\fBpcregrep\fP uses an internal buffer to hold a "window" on the file it is -scanning, in order to be able to output "before" and "after" lines when it -finds a match. The size of the buffer is controlled by a parameter whose -default value is 20K. The buffer itself is three times this size, but because -of the way it is used for holding "before" lines, the longest line that is -guaranteed to be processable is the parameter size. You can change the default -parameter value by adding, for example, -.sp - --with-pcregrep-bufsize=50K -.sp -to the \fBconfigure\fP command. The caller of \fPpcregrep\fP can, however, -override this value by specifying a run-time option. -. -. -.SH "PCRETEST OPTION FOR LIBREADLINE SUPPORT" -.rs -.sp -If you add -.sp - --enable-pcretest-libreadline -.sp -to the \fBconfigure\fP command, \fBpcretest\fP is linked with the -\fBlibreadline\fP library, and when its input is from a terminal, it reads it -using the \fBreadline()\fP function. This provides line-editing and history -facilities. Note that \fBlibreadline\fP is GPL-licensed, so if you distribute a -binary of \fBpcretest\fP linked in this way, there may be licensing issues. -.P -Setting this option causes the \fB-lreadline\fP option to be added to the -\fBpcretest\fP build. In many operating environments with a sytem-installed -\fBlibreadline\fP this is sufficient. However, in some environments (e.g. -if an unmodified distribution version of readline is in use), some extra -configuration may be necessary. The INSTALL file for \fBlibreadline\fP says -this: -.sp - "Readline uses the termcap functions, but does not link with the - termcap or curses library itself, allowing applications which link - with readline the to choose an appropriate library." -.sp -If your environment has not been set up so that an appropriate library is -automatically included, you may need to add something like -.sp - LIBS="-ncurses" -.sp -immediately before the \fBconfigure\fP command. -. -. -.SH "DEBUGGING WITH VALGRIND SUPPORT" -.rs -.sp -By adding the -.sp - --enable-valgrind -.sp -option to to the \fBconfigure\fP command, PCRE will use valgrind annotations -to mark certain memory regions as unaddressable. This allows it to detect -invalid memory accesses, and is mostly useful for debugging PCRE itself. -. -. -.SH "CODE COVERAGE REPORTING" -.rs -.sp -If your C compiler is gcc, you can build a version of PCRE that can generate a -code coverage report for its test suite. To enable this, you must install -\fBlcov\fP version 1.6 or above. Then specify -.sp - --enable-coverage -.sp -to the \fBconfigure\fP command and build PCRE in the usual way. -.P -Note that using \fBccache\fP (a caching C compiler) is incompatible with code -coverage reporting. If you have configured \fBccache\fP to run automatically -on your system, you must set the environment variable -.sp - CCACHE_DISABLE=1 -.sp -before running \fBmake\fP to build PCRE, so that \fBccache\fP is not used. -.P -When --enable-coverage is used, the following addition targets are added to the -\fIMakefile\fP: -.sp - make coverage -.sp -This creates a fresh coverage report for the PCRE test suite. It is equivalent -to running "make coverage-reset", "make coverage-baseline", "make check", and -then "make coverage-report". -.sp - make coverage-reset -.sp -This zeroes the coverage counters, but does nothing else. -.sp - make coverage-baseline -.sp -This captures baseline coverage information. -.sp - make coverage-report -.sp -This creates the coverage report. -.sp - make coverage-clean-report -.sp -This removes the generated coverage report without cleaning the coverage data -itself. -.sp - make coverage-clean-data -.sp -This removes the captured coverage data without removing the coverage files -created at compile time (*.gcno). -.sp - make coverage-clean -.sp -This cleans all coverage data including the generated coverage report. For more -information about code coverage, see the \fBgcov\fP and \fBlcov\fP -documentation. -. -. -.SH "SEE ALSO" -.rs -.sp -\fBpcreapi\fP(3), \fBpcre16\fP, \fBpcre32\fP, \fBpcre_config\fP(3). -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 12 May 2013 -Copyright (c) 1997-2013 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcrecallout.3 b/plugins/Pcre16/docs/doc/pcrecallout.3 deleted file mode 100644 index 8ebc995952..0000000000 --- a/plugins/Pcre16/docs/doc/pcrecallout.3 +++ /dev/null @@ -1,255 +0,0 @@ -.TH PCRECALLOUT 3 "12 November 2013" "PCRE 8.34" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH SYNOPSIS -.rs -.sp -.B #include <pcre.h> -.PP -.SM -.B int (*pcre_callout)(pcre_callout_block *); -.PP -.B int (*pcre16_callout)(pcre16_callout_block *); -.PP -.B int (*pcre32_callout)(pcre32_callout_block *); -. -.SH DESCRIPTION -.rs -.sp -PCRE provides a feature called "callout", which is a means of temporarily -passing control to the caller of PCRE in the middle of pattern matching. The -caller of PCRE provides an external function by putting its entry point in the -global variable \fIpcre_callout\fP (\fIpcre16_callout\fP for the 16-bit -library, \fIpcre32_callout\fP for the 32-bit library). By default, this -variable contains NULL, which disables all calling out. -.P -Within a regular expression, (?C) indicates the points at which the external -function is to be called. Different callout points can be identified by putting -a number less than 256 after the letter C. The default value is zero. -For example, this pattern has two callout points: -.sp - (?C1)abc(?C2)def -.sp -If the PCRE_AUTO_CALLOUT option bit is set when a pattern is compiled, PCRE -automatically inserts callouts, all with number 255, before each item in the -pattern. For example, if PCRE_AUTO_CALLOUT is used with the pattern -.sp - A(\ed{2}|--) -.sp -it is processed as if it were -.sp -(?C255)A(?C255)((?C255)\ed{2}(?C255)|(?C255)-(?C255)-(?C255))(?C255) -.sp -Notice that there is a callout before and after each parenthesis and -alternation bar. If the pattern contains a conditional group whose condition is -an assertion, an automatic callout is inserted immediately before the -condition. Such a callout may also be inserted explicitly, for example: -.sp - (?(?C9)(?=a)ab|de) -.sp -This applies only to assertion conditions (because they are themselves -independent groups). -.P -Automatic callouts can be used for tracking the progress of pattern matching. -The -.\" HREF -\fBpcretest\fP -.\" -program has a pattern qualifier (/C) that sets automatic callouts; when it is -used, the output indicates how the pattern is being matched. This is useful -information when you are trying to optimize the performance of a particular -pattern. -. -. -.SH "MISSING CALLOUTS" -.rs -.sp -You should be aware that, because of optimizations in the way PCRE compiles and -matches patterns, callouts sometimes do not happen exactly as you might expect. -.P -At compile time, PCRE "auto-possessifies" repeated items when it knows that -what follows cannot be part of the repeat. For example, a+[bc] is compiled as -if it were a++[bc]. The \fBpcretest\fP output when this pattern is anchored and -then applied with automatic callouts to the string "aaaa" is: -.sp - --->aaaa - +0 ^ ^ - +1 ^ a+ - +3 ^ ^ [bc] - No match -.sp -This indicates that when matching [bc] fails, there is no backtracking into a+ -and therefore the callouts that would be taken for the backtracks do not occur. -You can disable the auto-possessify feature by passing PCRE_NO_AUTO_POSSESS -to \fBpcre_compile()\fP, or starting the pattern with (*NO_AUTO_POSSESS). If -this is done in \fBpcretest\fP (using the /O qualifier), the output changes to -this: -.sp - --->aaaa - +0 ^ ^ - +1 ^ a+ - +3 ^ ^ [bc] - +3 ^ ^ [bc] - +3 ^ ^ [bc] - +3 ^^ [bc] - No match -.sp -This time, when matching [bc] fails, the matcher backtracks into a+ and tries -again, repeatedly, until a+ itself fails. -.P -Other optimizations that provide fast "no match" results also affect callouts. -For example, if the pattern is -.sp - ab(?C4)cd -.sp -PCRE knows that any matching string must contain the letter "d". If the subject -string is "abyz", the lack of "d" means that matching doesn't ever start, and -the callout is never reached. However, with "abyd", though the result is still -no match, the callout is obeyed. -.P -If the pattern is studied, PCRE knows the minimum length of a matching string, -and will immediately give a "no match" return without actually running a match -if the subject is not long enough, or, for unanchored patterns, if it has -been scanned far enough. -.P -You can disable these optimizations by passing the PCRE_NO_START_OPTIMIZE -option to the matching function, or by starting the pattern with -(*NO_START_OPT). This slows down the matching process, but does ensure that -callouts such as the example above are obeyed. -. -. -.SH "THE CALLOUT INTERFACE" -.rs -.sp -During matching, when PCRE reaches a callout point, the external function -defined by \fIpcre_callout\fP or \fIpcre[16|32]_callout\fP is called (if it is -set). This applies to both normal and DFA matching. The only argument to the -callout function is a pointer to a \fBpcre_callout\fP or -\fBpcre[16|32]_callout\fP block. These structures contains the following -fields: -.sp - int \fIversion\fP; - int \fIcallout_number\fP; - int *\fIoffset_vector\fP; - const char *\fIsubject\fP; (8-bit version) - PCRE_SPTR16 \fIsubject\fP; (16-bit version) - PCRE_SPTR32 \fIsubject\fP; (32-bit version) - int \fIsubject_length\fP; - int \fIstart_match\fP; - int \fIcurrent_position\fP; - int \fIcapture_top\fP; - int \fIcapture_last\fP; - void *\fIcallout_data\fP; - int \fIpattern_position\fP; - int \fInext_item_length\fP; - const unsigned char *\fImark\fP; (8-bit version) - const PCRE_UCHAR16 *\fImark\fP; (16-bit version) - const PCRE_UCHAR32 *\fImark\fP; (32-bit version) -.sp -The \fIversion\fP field is an integer containing the version number of the -block format. The initial version was 0; the current version is 2. The version -number will change again in future if additional fields are added, but the -intention is never to remove any of the existing fields. -.P -The \fIcallout_number\fP field contains the number of the callout, as compiled -into the pattern (that is, the number after ?C for manual callouts, and 255 for -automatically generated callouts). -.P -The \fIoffset_vector\fP field is a pointer to the vector of offsets that was -passed by the caller to the matching function. When \fBpcre_exec()\fP or -\fBpcre[16|32]_exec()\fP is used, the contents can be inspected, in order to -extract substrings that have been matched so far, in the same way as for -extracting substrings after a match has completed. For the DFA matching -functions, this field is not useful. -.P -The \fIsubject\fP and \fIsubject_length\fP fields contain copies of the values -that were passed to the matching function. -.P -The \fIstart_match\fP field normally contains the offset within the subject at -which the current match attempt started. However, if the escape sequence \eK -has been encountered, this value is changed to reflect the modified starting -point. If the pattern is not anchored, the callout function may be called -several times from the same point in the pattern for different starting points -in the subject. -.P -The \fIcurrent_position\fP field contains the offset within the subject of the -current match pointer. -.P -When the \fBpcre_exec()\fP or \fBpcre[16|32]_exec()\fP is used, the -\fIcapture_top\fP field contains one more than the number of the highest -numbered captured substring so far. If no substrings have been captured, the -value of \fIcapture_top\fP is one. This is always the case when the DFA -functions are used, because they do not support captured substrings. -.P -The \fIcapture_last\fP field contains the number of the most recently captured -substring. However, when a recursion exits, the value reverts to what it was -outside the recursion, as do the values of all captured substrings. If no -substrings have been captured, the value of \fIcapture_last\fP is -1. This is -always the case for the DFA matching functions. -.P -The \fIcallout_data\fP field contains a value that is passed to a matching -function specifically so that it can be passed back in callouts. It is passed -in the \fIcallout_data\fP field of a \fBpcre_extra\fP or \fBpcre[16|32]_extra\fP -data structure. If no such data was passed, the value of \fIcallout_data\fP in -a callout block is NULL. There is a description of the \fBpcre_extra\fP -structure in the -.\" HREF -\fBpcreapi\fP -.\" -documentation. -.P -The \fIpattern_position\fP field is present from version 1 of the callout -structure. It contains the offset to the next item to be matched in the pattern -string. -.P -The \fInext_item_length\fP field is present from version 1 of the callout -structure. It contains the length of the next item to be matched in the pattern -string. When the callout immediately precedes an alternation bar, a closing -parenthesis, or the end of the pattern, the length is zero. When the callout -precedes an opening parenthesis, the length is that of the entire subpattern. -.P -The \fIpattern_position\fP and \fInext_item_length\fP fields are intended to -help in distinguishing between different automatic callouts, which all have the -same callout number. However, they are set for all callouts. -.P -The \fImark\fP field is present from version 2 of the callout structure. In -callouts from \fBpcre_exec()\fP or \fBpcre[16|32]_exec()\fP it contains a -pointer to the zero-terminated name of the most recently passed (*MARK), -(*PRUNE), or (*THEN) item in the match, or NULL if no such items have been -passed. Instances of (*PRUNE) or (*THEN) without a name do not obliterate a -previous (*MARK). In callouts from the DFA matching functions this field always -contains NULL. -. -. -.SH "RETURN VALUES" -.rs -.sp -The external callout function returns an integer to PCRE. If the value is zero, -matching proceeds as normal. If the value is greater than zero, matching fails -at the current point, but the testing of other matching possibilities goes -ahead, just as if a lookahead assertion had failed. If the value is less than -zero, the match is abandoned, the matching function returns the negative value. -.P -Negative values should normally be chosen from the set of PCRE_ERROR_xxx -values. In particular, PCRE_ERROR_NOMATCH forces a standard "no match" failure. -The error number PCRE_ERROR_CALLOUT is reserved for use by callout functions; -it will never be used by PCRE itself. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 12 November 2013 -Copyright (c) 1997-2013 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcrecompat.3 b/plugins/Pcre16/docs/doc/pcrecompat.3 deleted file mode 100644 index 0cc4019823..0000000000 --- a/plugins/Pcre16/docs/doc/pcrecompat.3 +++ /dev/null @@ -1,200 +0,0 @@ -.TH PCRECOMPAT 3 "10 November 2013" "PCRE 8.34" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH "DIFFERENCES BETWEEN PCRE AND PERL" -.rs -.sp -This document describes the differences in the ways that PCRE and Perl handle -regular expressions. The differences described here are with respect to Perl -versions 5.10 and above. -.P -1. PCRE has only a subset of Perl's Unicode support. Details of what it does -have are given in the -.\" HREF -\fBpcreunicode\fP -.\" -page. -.P -2. PCRE allows repeat quantifiers only on parenthesized assertions, but they do -not mean what you might think. For example, (?!a){3} does not assert that the -next three characters are not "a". It just asserts that the next character is -not "a" three times (in principle: PCRE optimizes this to run the assertion -just once). Perl allows repeat quantifiers on other assertions such as \eb, but -these do not seem to have any use. -.P -3. Capturing subpatterns that occur inside negative lookahead assertions are -counted, but their entries in the offsets vector are never set. Perl sometimes -(but not always) sets its numerical variables from inside negative assertions. -.P -4. Though binary zero characters are supported in the subject string, they are -not allowed in a pattern string because it is passed as a normal C string, -terminated by zero. The escape sequence \e0 can be used in the pattern to -represent a binary zero. -.P -5. The following Perl escape sequences are not supported: \el, \eu, \eL, -\eU, and \eN when followed by a character name or Unicode value. (\eN on its -own, matching a non-newline character, is supported.) In fact these are -implemented by Perl's general string-handling and are not part of its pattern -matching engine. If any of these are encountered by PCRE, an error is -generated by default. However, if the PCRE_JAVASCRIPT_COMPAT option is set, -\eU and \eu are interpreted as JavaScript interprets them. -.P -6. The Perl escape sequences \ep, \eP, and \eX are supported only if PCRE is -built with Unicode character property support. The properties that can be -tested with \ep and \eP are limited to the general category properties such as -Lu and Nd, script names such as Greek or Han, and the derived properties Any -and L&. PCRE does support the Cs (surrogate) property, which Perl does not; the -Perl documentation says "Because Perl hides the need for the user to understand -the internal representation of Unicode characters, there is no need to -implement the somewhat messy concept of surrogates." -.P -7. PCRE does support the \eQ...\eE escape for quoting substrings. Characters in -between are treated as literals. This is slightly different from Perl in that $ -and @ are also handled as literals inside the quotes. In Perl, they cause -variable interpolation (but of course PCRE does not have variables). Note the -following examples: -.sp - Pattern PCRE matches Perl matches -.sp -.\" JOIN - \eQabc$xyz\eE abc$xyz abc followed by the - contents of $xyz - \eQabc\e$xyz\eE abc\e$xyz abc\e$xyz - \eQabc\eE\e$\eQxyz\eE abc$xyz abc$xyz -.sp -The \eQ...\eE sequence is recognized both inside and outside character classes. -.P -8. Fairly obviously, PCRE does not support the (?{code}) and (??{code}) -constructions. However, there is support for recursive patterns. This is not -available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE "callout" -feature allows an external function to be called during pattern matching. See -the -.\" HREF -\fBpcrecallout\fP -.\" -documentation for details. -.P -9. Subpatterns that are called as subroutines (whether or not recursively) are -always treated as atomic groups in PCRE. This is like Python, but unlike Perl. -Captured values that are set outside a subroutine call can be reference from -inside in PCRE, but not in Perl. There is a discussion that explains these -differences in more detail in the -.\" HTML <a href="pcrepattern.html#recursiondifference"> -.\" </a> -section on recursion differences from Perl -.\" -in the -.\" HREF -\fBpcrepattern\fP -.\" -page. -.P -10. If any of the backtracking control verbs are used in a subpattern that is -called as a subroutine (whether or not recursively), their effect is confined -to that subpattern; it does not extend to the surrounding pattern. This is not -always the case in Perl. In particular, if (*THEN) is present in a group that -is called as a subroutine, its action is limited to that group, even if the -group does not contain any | characters. Note that such subpatterns are -processed as anchored at the point where they are tested. -.P -11. If a pattern contains more than one backtracking control verb, the first -one that is backtracked onto acts. For example, in the pattern -A(*COMMIT)B(*PRUNE)C a failure in B triggers (*COMMIT), but a failure in C -triggers (*PRUNE). Perl's behaviour is more complex; in many cases it is the -same as PCRE, but there are examples where it differs. -.P -12. Most backtracking verbs in assertions have their normal actions. They are -not confined to the assertion. -.P -13. There are some differences that are concerned with the settings of captured -strings when part of a pattern is repeated. For example, matching "aba" against -the pattern /^(a(b)?)+$/ in Perl leaves $2 unset, but in PCRE it is set to "b". -.P -14. PCRE's handling of duplicate subpattern numbers and duplicate subpattern -names is not as general as Perl's. This is a consequence of the fact the PCRE -works internally just with numbers, using an external table to translate -between numbers and names. In particular, a pattern such as (?|(?<a>A)|(?<b)B), -where the two capturing parentheses have the same number but different names, -is not supported, and causes an error at compile time. If it were allowed, it -would not be possible to distinguish which parentheses matched, because both -names map to capturing subpattern number 1. To avoid this confusing situation, -an error is given at compile time. -.P -15. Perl recognizes comments in some places that PCRE does not, for example, -between the ( and ? at the start of a subpattern. If the /x modifier is set, -Perl allows white space between ( and ? (though current Perls warn that this is -deprecated) but PCRE never does, even if the PCRE_EXTENDED option is set. -.P -16. Perl, when in warning mode, gives warnings for character classes such as -[A-\ed] or [a-[:digit:]]. It then treats the hyphens as literals. PCRE has no -warning features, so it gives an error in these cases because they are almost -certainly user mistakes. -.P -17. In PCRE, the upper/lower case character properties Lu and Ll are not -affected when case-independent matching is specified. For example, \ep{Lu} -always matches an upper case letter. I think Perl has changed in this respect; -in the release at the time of writing (5.16), \ep{Lu} and \ep{Ll} match all -letters, regardless of case, when case independence is specified. -.P -18. PCRE provides some extensions to the Perl regular expression facilities. -Perl 5.10 includes new features that are not in earlier versions of Perl, some -of which (such as named parentheses) have been in PCRE for some time. This list -is with respect to Perl 5.10: -.sp -(a) Although lookbehind assertions in PCRE must match fixed length strings, -each alternative branch of a lookbehind assertion can match a different length -of string. Perl requires them all to have the same length. -.sp -(b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $ -meta-character matches only at the very end of the string. -.sp -(c) If PCRE_EXTRA is set, a backslash followed by a letter with no special -meaning is faulted. Otherwise, like Perl, the backslash is quietly ignored. -(Perl can be made to issue a warning.) -.sp -(d) If PCRE_UNGREEDY is set, the greediness of the repetition quantifiers is -inverted, that is, by default they are not greedy, but if followed by a -question mark they are. -.sp -(e) PCRE_ANCHORED can be used at matching time to force a pattern to be tried -only at the first matching position in the subject string. -.sp -(f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, and -PCRE_NO_AUTO_CAPTURE options for \fBpcre_exec()\fP have no Perl equivalents. -.sp -(g) The \eR escape sequence can be restricted to match only CR, LF, or CRLF -by the PCRE_BSR_ANYCRLF option. -.sp -(h) The callout facility is PCRE-specific. -.sp -(i) The partial matching facility is PCRE-specific. -.sp -(j) Patterns compiled by PCRE can be saved and re-used at a later time, even on -different hosts that have the other endianness. However, this does not apply to -optimized data created by the just-in-time compiler. -.sp -(k) The alternative matching functions (\fBpcre_dfa_exec()\fP, -\fBpcre16_dfa_exec()\fP and \fBpcre32_dfa_exec()\fP,) match in a different way -and are not Perl-compatible. -.sp -(l) PCRE recognizes some special sequences such as (*CR) at the start of -a pattern that set overall options that cannot be changed within the pattern. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 10 November 2013 -Copyright (c) 1997-2013 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcrecpp.3 b/plugins/Pcre16/docs/doc/pcrecpp.3 deleted file mode 100644 index fbddd86ab3..0000000000 --- a/plugins/Pcre16/docs/doc/pcrecpp.3 +++ /dev/null @@ -1,348 +0,0 @@ -.TH PCRECPP 3 "08 January 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions. -.SH "SYNOPSIS OF C++ WRAPPER" -.rs -.sp -.B #include <pcrecpp.h> -. -.SH DESCRIPTION -.rs -.sp -The C++ wrapper for PCRE was provided by Google Inc. Some additional -functionality was added by Giuseppe Maxia. This brief man page was constructed -from the notes in the \fIpcrecpp.h\fP file, which should be consulted for -further details. Note that the C++ wrapper supports only the original 8-bit -PCRE library. There is no 16-bit or 32-bit support at present. -. -. -.SH "MATCHING INTERFACE" -.rs -.sp -The "FullMatch" operation checks that supplied text matches a supplied pattern -exactly. If pointer arguments are supplied, it copies matched sub-strings that -match sub-patterns into them. -.sp - Example: successful match - pcrecpp::RE re("h.*o"); - re.FullMatch("hello"); -.sp - Example: unsuccessful match (requires full match): - pcrecpp::RE re("e"); - !re.FullMatch("hello"); -.sp - Example: creating a temporary RE object: - pcrecpp::RE("h.*o").FullMatch("hello"); -.sp -You can pass in a "const char*" or a "string" for "text". The examples below -tend to use a const char*. You can, as in the different examples above, store -the RE object explicitly in a variable or use a temporary RE object. The -examples below use one mode or the other arbitrarily. Either could correctly be -used for any of these examples. -.P -You must supply extra pointer arguments to extract matched subpieces. -.sp - Example: extracts "ruby" into "s" and 1234 into "i" - int i; - string s; - pcrecpp::RE re("(\e\ew+):(\e\ed+)"); - re.FullMatch("ruby:1234", &s, &i); -.sp - Example: does not try to extract any extra sub-patterns - re.FullMatch("ruby:1234", &s); -.sp - Example: does not try to extract into NULL - re.FullMatch("ruby:1234", NULL, &i); -.sp - Example: integer overflow causes failure - !re.FullMatch("ruby:1234567891234", NULL, &i); -.sp - Example: fails because there aren't enough sub-patterns: - !pcrecpp::RE("\e\ew+:\e\ed+").FullMatch("ruby:1234", &s); -.sp - Example: fails because string cannot be stored in integer - !pcrecpp::RE("(.*)").FullMatch("ruby", &i); -.sp -The provided pointer arguments can be pointers to any scalar numeric -type, or one of: -.sp - string (matched piece is copied to string) - StringPiece (StringPiece is mutated to point to matched piece) - T (where "bool T::ParseFrom(const char*, int)" exists) - NULL (the corresponding matched sub-pattern is not copied) -.sp -The function returns true iff all of the following conditions are satisfied: -.sp - a. "text" matches "pattern" exactly; -.sp - b. The number of matched sub-patterns is >= number of supplied - pointers; -.sp - c. The "i"th argument has a suitable type for holding the - string captured as the "i"th sub-pattern. If you pass in - void * NULL for the "i"th argument, or a non-void * NULL - of the correct type, or pass fewer arguments than the - number of sub-patterns, "i"th captured sub-pattern is - ignored. -.sp -CAVEAT: An optional sub-pattern that does not exist in the matched -string is assigned the empty string. Therefore, the following will -return false (because the empty string is not a valid number): -.sp - int number; - pcrecpp::RE::FullMatch("abc", "[a-z]+(\e\ed+)?", &number); -.sp -The matching interface supports at most 16 arguments per call. -If you need more, consider using the more general interface -\fBpcrecpp::RE::DoMatch\fP. See \fBpcrecpp.h\fP for the signature for -\fBDoMatch\fP. -.P -NOTE: Do not use \fBno_arg\fP, which is used internally to mark the end of a -list of optional arguments, as a placeholder for missing arguments, as this can -lead to segfaults. -. -. -.SH "QUOTING METACHARACTERS" -.rs -.sp -You can use the "QuoteMeta" operation to insert backslashes before all -potentially meaningful characters in a string. The returned string, used as a -regular expression, will exactly match the original string. -.sp - Example: - string quoted = RE::QuoteMeta(unquoted); -.sp -Note that it's legal to escape a character even if it has no special meaning in -a regular expression -- so this function does that. (This also makes it -identical to the perl function of the same name; see "perldoc -f quotemeta".) -For example, "1.5-2.0?" becomes "1\e.5\e-2\e.0\e?". -. -.SH "PARTIAL MATCHES" -.rs -.sp -You can use the "PartialMatch" operation when you want the pattern -to match any substring of the text. -.sp - Example: simple search for a string: - pcrecpp::RE("ell").PartialMatch("hello"); -.sp - Example: find first number in a string: - int number; - pcrecpp::RE re("(\e\ed+)"); - re.PartialMatch("x*100 + 20", &number); - assert(number == 100); -. -. -.SH "UTF-8 AND THE MATCHING INTERFACE" -.rs -.sp -By default, pattern and text are plain text, one byte per character. The UTF8 -flag, passed to the constructor, causes both pattern and string to be treated -as UTF-8 text, still a byte stream but potentially multiple bytes per -character. In practice, the text is likelier to be UTF-8 than the pattern, but -the match returned may depend on the UTF8 flag, so always use it when matching -UTF8 text. For example, "." will match one byte normally but with UTF8 set may -match up to three bytes of a multi-byte character. -.sp - Example: - pcrecpp::RE_Options options; - options.set_utf8(); - pcrecpp::RE re(utf8_pattern, options); - re.FullMatch(utf8_string); -.sp - Example: using the convenience function UTF8(): - pcrecpp::RE re(utf8_pattern, pcrecpp::UTF8()); - re.FullMatch(utf8_string); -.sp -NOTE: The UTF8 flag is ignored if pcre was not configured with the - --enable-utf8 flag. -. -. -.SH "PASSING MODIFIERS TO THE REGULAR EXPRESSION ENGINE" -.rs -.sp -PCRE defines some modifiers to change the behavior of the regular expression -engine. The C++ wrapper defines an auxiliary class, RE_Options, as a vehicle to -pass such modifiers to a RE class. Currently, the following modifiers are -supported: -.sp - modifier description Perl corresponding -.sp - PCRE_CASELESS case insensitive match /i - PCRE_MULTILINE multiple lines match /m - PCRE_DOTALL dot matches newlines /s - PCRE_DOLLAR_ENDONLY $ matches only at end N/A - PCRE_EXTRA strict escape parsing N/A - PCRE_EXTENDED ignore white spaces /x - PCRE_UTF8 handles UTF8 chars built-in - PCRE_UNGREEDY reverses * and *? N/A - PCRE_NO_AUTO_CAPTURE disables capturing parens N/A (*) -.sp -(*) Both Perl and PCRE allow non capturing parentheses by means of the -"?:" modifier within the pattern itself. e.g. (?:ab|cd) does not -capture, while (ab|cd) does. -.P -For a full account on how each modifier works, please check the -PCRE API reference page. -.P -For each modifier, there are two member functions whose name is made -out of the modifier in lowercase, without the "PCRE_" prefix. For -instance, PCRE_CASELESS is handled by -.sp - bool caseless() -.sp -which returns true if the modifier is set, and -.sp - RE_Options & set_caseless(bool) -.sp -which sets or unsets the modifier. Moreover, PCRE_EXTRA_MATCH_LIMIT can be -accessed through the \fBset_match_limit()\fP and \fBmatch_limit()\fP member -functions. Setting \fImatch_limit\fP to a non-zero value will limit the -execution of pcre to keep it from doing bad things like blowing the stack or -taking an eternity to return a result. A value of 5000 is good enough to stop -stack blowup in a 2MB thread stack. Setting \fImatch_limit\fP to zero disables -match limiting. Alternatively, you can call \fBmatch_limit_recursion()\fP -which uses PCRE_EXTRA_MATCH_LIMIT_RECURSION to limit how much PCRE -recurses. \fBmatch_limit()\fP limits the number of matches PCRE does; -\fBmatch_limit_recursion()\fP limits the depth of internal recursion, and -therefore the amount of stack that is used. -.P -Normally, to pass one or more modifiers to a RE class, you declare -a \fIRE_Options\fP object, set the appropriate options, and pass this -object to a RE constructor. Example: -.sp - RE_Options opt; - opt.set_caseless(true); - if (RE("HELLO", opt).PartialMatch("hello world")) ... -.sp -RE_options has two constructors. The default constructor takes no arguments and -creates a set of flags that are off by default. The optional parameter -\fIoption_flags\fP is to facilitate transfer of legacy code from C programs. -This lets you do -.sp - RE(pattern, - RE_Options(PCRE_CASELESS|PCRE_MULTILINE)).PartialMatch(str); -.sp -However, new code is better off doing -.sp - RE(pattern, - RE_Options().set_caseless(true).set_multiline(true)) - .PartialMatch(str); -.sp -If you are going to pass one of the most used modifiers, there are some -convenience functions that return a RE_Options class with the -appropriate modifier already set: \fBCASELESS()\fP, \fBUTF8()\fP, -\fBMULTILINE()\fP, \fBDOTALL\fP(), and \fBEXTENDED()\fP. -.P -If you need to set several options at once, and you don't want to go through -the pains of declaring a RE_Options object and setting several options, there -is a parallel method that give you such ability on the fly. You can concatenate -several \fBset_xxxxx()\fP member functions, since each of them returns a -reference to its class object. For example, to pass PCRE_CASELESS, -PCRE_EXTENDED, and PCRE_MULTILINE to a RE with one statement, you may write: -.sp - RE(" ^ xyz \e\es+ .* blah$", - RE_Options() - .set_caseless(true) - .set_extended(true) - .set_multiline(true)).PartialMatch(sometext); -.sp -. -. -.SH "SCANNING TEXT INCREMENTALLY" -.rs -.sp -The "Consume" operation may be useful if you want to repeatedly -match regular expressions at the front of a string and skip over -them as they match. This requires use of the "StringPiece" type, -which represents a sub-range of a real string. Like RE, StringPiece -is defined in the pcrecpp namespace. -.sp - Example: read lines of the form "var = value" from a string. - string contents = ...; // Fill string somehow - pcrecpp::StringPiece input(contents); // Wrap in a StringPiece -.sp - string var; - int value; - pcrecpp::RE re("(\e\ew+) = (\e\ed+)\en"); - while (re.Consume(&input, &var, &value)) { - ...; - } -.sp -Each successful call to "Consume" will set "var/value", and also -advance "input" so it points past the matched text. -.P -The "FindAndConsume" operation is similar to "Consume" but does not -anchor your match at the beginning of the string. For example, you -could extract all words from a string by repeatedly calling -.sp - pcrecpp::RE("(\e\ew+)").FindAndConsume(&input, &word) -. -. -.SH "PARSING HEX/OCTAL/C-RADIX NUMBERS" -.rs -.sp -By default, if you pass a pointer to a numeric value, the -corresponding text is interpreted as a base-10 number. You can -instead wrap the pointer with a call to one of the operators Hex(), -Octal(), or CRadix() to interpret the text in another base. The -CRadix operator interprets C-style "0" (base-8) and "0x" (base-16) -prefixes, but defaults to base-10. -.sp - Example: - int a, b, c, d; - pcrecpp::RE re("(.*) (.*) (.*) (.*)"); - re.FullMatch("100 40 0100 0x40", - pcrecpp::Octal(&a), pcrecpp::Hex(&b), - pcrecpp::CRadix(&c), pcrecpp::CRadix(&d)); -.sp -will leave 64 in a, b, c, and d. -. -. -.SH "REPLACING PARTS OF STRINGS" -.rs -.sp -You can replace the first match of "pattern" in "str" with "rewrite". -Within "rewrite", backslash-escaped digits (\e1 to \e9) can be -used to insert text matching corresponding parenthesized group -from the pattern. \e0 in "rewrite" refers to the entire matching -text. For example: -.sp - string s = "yabba dabba doo"; - pcrecpp::RE("b+").Replace("d", &s); -.sp -will leave "s" containing "yada dabba doo". The result is true if the pattern -matches and a replacement occurs, false otherwise. -.P -\fBGlobalReplace\fP is like \fBReplace\fP except that it replaces all -occurrences of the pattern in the string with the rewrite. Replacements are -not subject to re-matching. For example: -.sp - string s = "yabba dabba doo"; - pcrecpp::RE("b+").GlobalReplace("d", &s); -.sp -will leave "s" containing "yada dada doo". It returns the number of -replacements made. -.P -\fBExtract\fP is like \fBReplace\fP, except that if the pattern matches, -"rewrite" is copied into "out" (an additional argument) with substitutions. -The non-matching portions of "text" are ignored. Returns true iff a match -occurred and the extraction happened successfully; if no match occurs, the -string is left unaffected. -. -. -.SH AUTHOR -.rs -.sp -.nf -The C++ wrapper was contributed by Google Inc. -Copyright (c) 2007 Google Inc. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 08 January 2012 -.fi diff --git a/plugins/Pcre16/docs/doc/pcredemo.3 b/plugins/Pcre16/docs/doc/pcredemo.3 deleted file mode 100644 index 194629b1fa..0000000000 --- a/plugins/Pcre16/docs/doc/pcredemo.3 +++ /dev/null @@ -1,424 +0,0 @@ -.\" Start example. -.de EX -. nr mE \\n(.f -. nf -. nh -. ft CW -.. -. -. -.\" End example. -.de EE -. ft \\n(mE -. fi -. hy \\n(HY -.. -. -.EX -/************************************************* -* PCRE DEMONSTRATION PROGRAM * -*************************************************/ - -/* This is a demonstration program to illustrate the most straightforward ways -of calling the PCRE regular expression library from a C program. See the -pcresample documentation for a short discussion ("man pcresample" if you have -the PCRE man pages installed). - -In Unix-like environments, if PCRE is installed in your standard system -libraries, you should be able to compile this program using this command: - -gcc -Wall pcredemo.c -lpcre -o pcredemo - -If PCRE is not installed in a standard place, it is likely to be installed with -support for the pkg-config mechanism. If you have pkg-config, you can compile -this program using this command: - -gcc -Wall pcredemo.c `pkg-config --cflags --libs libpcre` -o pcredemo - -If you do not have pkg-config, you may have to use this: - -gcc -Wall pcredemo.c -I/usr/local/include -L/usr/local/lib \e - -R/usr/local/lib -lpcre -o pcredemo - -Replace "/usr/local/include" and "/usr/local/lib" with wherever the include and -library files for PCRE are installed on your system. Only some operating -systems (e.g. Solaris) use the -R option. - -Building under Windows: - -If you want to statically link this program against a non-dll .a file, you must -define PCRE_STATIC before including pcre.h, otherwise the pcre_malloc() and -pcre_free() exported functions will be declared __declspec(dllimport), with -unwanted results. So in this environment, uncomment the following line. */ - -/* #define PCRE_STATIC */ - -#include <stdio.h> -#include <string.h> -#include <pcre.h> - -#define OVECCOUNT 30 /* should be a multiple of 3 */ - - -int main(int argc, char **argv) -{ -pcre *re; -const char *error; -char *pattern; -char *subject; -unsigned char *name_table; -unsigned int option_bits; -int erroffset; -int find_all; -int crlf_is_newline; -int namecount; -int name_entry_size; -int ovector[OVECCOUNT]; -int subject_length; -int rc, i; -int utf8; - - -/************************************************************************** -* First, sort out the command line. There is only one possible option at * -* the moment, "-g" to request repeated matching to find all occurrences, * -* like Perl's /g option. We set the variable find_all to a non-zero value * -* if the -g option is present. Apart from that, there must be exactly two * -* arguments. * -**************************************************************************/ - -find_all = 0; -for (i = 1; i < argc; i++) - { - if (strcmp(argv[i], "-g") == 0) find_all = 1; - else break; - } - -/* After the options, we require exactly two arguments, which are the pattern, -and the subject string. */ - -if (argc - i != 2) - { - printf("Two arguments required: a regex and a subject string\en"); - return 1; - } - -pattern = argv[i]; -subject = argv[i+1]; -subject_length = (int)strlen(subject); - - -/************************************************************************* -* Now we are going to compile the regular expression pattern, and handle * -* and errors that are detected. * -*************************************************************************/ - -re = pcre_compile( - pattern, /* the pattern */ - 0, /* default options */ - &error, /* for error message */ - &erroffset, /* for error offset */ - NULL); /* use default character tables */ - -/* Compilation failed: print the error message and exit */ - -if (re == NULL) - { - printf("PCRE compilation failed at offset %d: %s\en", erroffset, error); - return 1; - } - - -/************************************************************************* -* If the compilation succeeded, we call PCRE again, in order to do a * -* pattern match against the subject string. This does just ONE match. If * -* further matching is needed, it will be done below. * -*************************************************************************/ - -rc = pcre_exec( - re, /* the compiled pattern */ - NULL, /* no extra data - we didn't study the pattern */ - subject, /* the subject string */ - subject_length, /* the length of the subject */ - 0, /* start at offset 0 in the subject */ - 0, /* default options */ - ovector, /* output vector for substring information */ - OVECCOUNT); /* number of elements in the output vector */ - -/* Matching failed: handle error cases */ - -if (rc < 0) - { - switch(rc) - { - case PCRE_ERROR_NOMATCH: printf("No match\en"); break; - /* - Handle other special cases if you like - */ - default: printf("Matching error %d\en", rc); break; - } - pcre_free(re); /* Release memory used for the compiled pattern */ - return 1; - } - -/* Match succeded */ - -printf("\enMatch succeeded at offset %d\en", ovector[0]); - - -/************************************************************************* -* We have found the first match within the subject string. If the output * -* vector wasn't big enough, say so. Then output any substrings that were * -* captured. * -*************************************************************************/ - -/* The output vector wasn't big enough */ - -if (rc == 0) - { - rc = OVECCOUNT/3; - printf("ovector only has room for %d captured substrings\en", rc - 1); - } - -/* Show substrings stored in the output vector by number. Obviously, in a real -application you might want to do things other than print them. */ - -for (i = 0; i < rc; i++) - { - char *substring_start = subject + ovector[2*i]; - int substring_length = ovector[2*i+1] - ovector[2*i]; - printf("%2d: %.*s\en", i, substring_length, substring_start); - } - - -/************************************************************************** -* That concludes the basic part of this demonstration program. We have * -* compiled a pattern, and performed a single match. The code that follows * -* shows first how to access named substrings, and then how to code for * -* repeated matches on the same subject. * -**************************************************************************/ - -/* See if there are any named substrings, and if so, show them by name. First -we have to extract the count of named parentheses from the pattern. */ - -(void)pcre_fullinfo( - re, /* the compiled pattern */ - NULL, /* no extra data - we didn't study the pattern */ - PCRE_INFO_NAMECOUNT, /* number of named substrings */ - &namecount); /* where to put the answer */ - -if (namecount <= 0) printf("No named substrings\en"); else - { - unsigned char *tabptr; - printf("Named substrings\en"); - - /* Before we can access the substrings, we must extract the table for - translating names to numbers, and the size of each entry in the table. */ - - (void)pcre_fullinfo( - re, /* the compiled pattern */ - NULL, /* no extra data - we didn't study the pattern */ - PCRE_INFO_NAMETABLE, /* address of the table */ - &name_table); /* where to put the answer */ - - (void)pcre_fullinfo( - re, /* the compiled pattern */ - NULL, /* no extra data - we didn't study the pattern */ - PCRE_INFO_NAMEENTRYSIZE, /* size of each entry in the table */ - &name_entry_size); /* where to put the answer */ - - /* Now we can scan the table and, for each entry, print the number, the name, - and the substring itself. */ - - tabptr = name_table; - for (i = 0; i < namecount; i++) - { - int n = (tabptr[0] << 8) | tabptr[1]; - printf("(%d) %*s: %.*s\en", n, name_entry_size - 3, tabptr + 2, - ovector[2*n+1] - ovector[2*n], subject + ovector[2*n]); - tabptr += name_entry_size; - } - } - - -/************************************************************************* -* If the "-g" option was given on the command line, we want to continue * -* to search for additional matches in the subject string, in a similar * -* way to the /g option in Perl. This turns out to be trickier than you * -* might think because of the possibility of matching an empty string. * -* What happens is as follows: * -* * -* If the previous match was NOT for an empty string, we can just start * -* the next match at the end of the previous one. * -* * -* If the previous match WAS for an empty string, we can't do that, as it * -* would lead to an infinite loop. Instead, a special call of pcre_exec() * -* is made with the PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED flags set. * -* The first of these tells PCRE that an empty string at the start of the * -* subject is not a valid match; other possibilities must be tried. The * -* second flag restricts PCRE to one match attempt at the initial string * -* position. If this match succeeds, an alternative to the empty string * -* match has been found, and we can print it and proceed round the loop, * -* advancing by the length of whatever was found. If this match does not * -* succeed, we still stay in the loop, advancing by just one character. * -* In UTF-8 mode, which can be set by (*UTF8) in the pattern, this may be * -* more than one byte. * -* * -* However, there is a complication concerned with newlines. When the * -* newline convention is such that CRLF is a valid newline, we must * -* advance by two characters rather than one. The newline convention can * -* be set in the regex by (*CR), etc.; if not, we must find the default. * -*************************************************************************/ - -if (!find_all) /* Check for -g */ - { - pcre_free(re); /* Release the memory used for the compiled pattern */ - return 0; /* Finish unless -g was given */ - } - -/* Before running the loop, check for UTF-8 and whether CRLF is a valid newline -sequence. First, find the options with which the regex was compiled; extract -the UTF-8 state, and mask off all but the newline options. */ - -(void)pcre_fullinfo(re, NULL, PCRE_INFO_OPTIONS, &option_bits); -utf8 = option_bits & PCRE_UTF8; -option_bits &= PCRE_NEWLINE_CR|PCRE_NEWLINE_LF|PCRE_NEWLINE_CRLF| - PCRE_NEWLINE_ANY|PCRE_NEWLINE_ANYCRLF; - -/* If no newline options were set, find the default newline convention from the -build configuration. */ - -if (option_bits == 0) - { - int d; - (void)pcre_config(PCRE_CONFIG_NEWLINE, &d); - /* Note that these values are always the ASCII ones, even in - EBCDIC environments. CR = 13, NL = 10. */ - option_bits = (d == 13)? PCRE_NEWLINE_CR : - (d == 10)? PCRE_NEWLINE_LF : - (d == (13<<8 | 10))? PCRE_NEWLINE_CRLF : - (d == -2)? PCRE_NEWLINE_ANYCRLF : - (d == -1)? PCRE_NEWLINE_ANY : 0; - } - -/* See if CRLF is a valid newline sequence. */ - -crlf_is_newline = - option_bits == PCRE_NEWLINE_ANY || - option_bits == PCRE_NEWLINE_CRLF || - option_bits == PCRE_NEWLINE_ANYCRLF; - -/* Loop for second and subsequent matches */ - -for (;;) - { - int options = 0; /* Normally no options */ - int start_offset = ovector[1]; /* Start at end of previous match */ - - /* If the previous match was for an empty string, we are finished if we are - at the end of the subject. Otherwise, arrange to run another match at the - same point to see if a non-empty match can be found. */ - - if (ovector[0] == ovector[1]) - { - if (ovector[0] == subject_length) break; - options = PCRE_NOTEMPTY_ATSTART | PCRE_ANCHORED; - } - - /* Run the next matching operation */ - - rc = pcre_exec( - re, /* the compiled pattern */ - NULL, /* no extra data - we didn't study the pattern */ - subject, /* the subject string */ - subject_length, /* the length of the subject */ - start_offset, /* starting offset in the subject */ - options, /* options */ - ovector, /* output vector for substring information */ - OVECCOUNT); /* number of elements in the output vector */ - - /* This time, a result of NOMATCH isn't an error. If the value in "options" - is zero, it just means we have found all possible matches, so the loop ends. - Otherwise, it means we have failed to find a non-empty-string match at a - point where there was a previous empty-string match. In this case, we do what - Perl does: advance the matching position by one character, and continue. We - do this by setting the "end of previous match" offset, because that is picked - up at the top of the loop as the point at which to start again. - - There are two complications: (a) When CRLF is a valid newline sequence, and - the current position is just before it, advance by an extra byte. (b) - Otherwise we must ensure that we skip an entire UTF-8 character if we are in - UTF-8 mode. */ - - if (rc == PCRE_ERROR_NOMATCH) - { - if (options == 0) break; /* All matches found */ - ovector[1] = start_offset + 1; /* Advance one byte */ - if (crlf_is_newline && /* If CRLF is newline & */ - start_offset < subject_length - 1 && /* we are at CRLF, */ - subject[start_offset] == '\er' && - subject[start_offset + 1] == '\en') - ovector[1] += 1; /* Advance by one more. */ - else if (utf8) /* Otherwise, ensure we */ - { /* advance a whole UTF-8 */ - while (ovector[1] < subject_length) /* character. */ - { - if ((subject[ovector[1]] & 0xc0) != 0x80) break; - ovector[1] += 1; - } - } - continue; /* Go round the loop again */ - } - - /* Other matching errors are not recoverable. */ - - if (rc < 0) - { - printf("Matching error %d\en", rc); - pcre_free(re); /* Release memory used for the compiled pattern */ - return 1; - } - - /* Match succeded */ - - printf("\enMatch succeeded again at offset %d\en", ovector[0]); - - /* The match succeeded, but the output vector wasn't big enough. */ - - if (rc == 0) - { - rc = OVECCOUNT/3; - printf("ovector only has room for %d captured substrings\en", rc - 1); - } - - /* As before, show substrings stored in the output vector by number, and then - also any named substrings. */ - - for (i = 0; i < rc; i++) - { - char *substring_start = subject + ovector[2*i]; - int substring_length = ovector[2*i+1] - ovector[2*i]; - printf("%2d: %.*s\en", i, substring_length, substring_start); - } - - if (namecount <= 0) printf("No named substrings\en"); else - { - unsigned char *tabptr = name_table; - printf("Named substrings\en"); - for (i = 0; i < namecount; i++) - { - int n = (tabptr[0] << 8) | tabptr[1]; - printf("(%d) %*s: %.*s\en", n, name_entry_size - 3, tabptr + 2, - ovector[2*n+1] - ovector[2*n], subject + ovector[2*n]); - tabptr += name_entry_size; - } - } - } /* End of loop to find second and subsequent matches */ - -printf("\en"); -pcre_free(re); /* Release memory used for the compiled pattern */ -return 0; -} - -/* End of pcredemo.c */ -.EE diff --git a/plugins/Pcre16/docs/doc/pcregrep.1 b/plugins/Pcre16/docs/doc/pcregrep.1 deleted file mode 100644 index 988667542f..0000000000 --- a/plugins/Pcre16/docs/doc/pcregrep.1 +++ /dev/null @@ -1,683 +0,0 @@ -.TH PCREGREP 1 "03 April 2014" "PCRE 8.35" -.SH NAME -pcregrep - a grep with Perl-compatible regular expressions. -.SH SYNOPSIS -.B pcregrep [options] [long options] [pattern] [path1 path2 ...] -. -.SH DESCRIPTION -.rs -.sp -\fBpcregrep\fP searches files for character patterns, in the same way as other -grep commands do, but it uses the PCRE regular expression library to support -patterns that are compatible with the regular expressions of Perl 5. See -.\" HREF -\fBpcresyntax\fP(3) -.\" -for a quick-reference summary of pattern syntax, or -.\" HREF -\fBpcrepattern\fP(3) -.\" -for a full description of the syntax and semantics of the regular expressions -that PCRE supports. -.P -Patterns, whether supplied on the command line or in a separate file, are given -without delimiters. For example: -.sp - pcregrep Thursday /etc/motd -.sp -If you attempt to use delimiters (for example, by surrounding a pattern with -slashes, as is common in Perl scripts), they are interpreted as part of the -pattern. Quotes can of course be used to delimit patterns on the command line -because they are interpreted by the shell, and indeed quotes are required if a -pattern contains white space or shell metacharacters. -.P -The first argument that follows any option settings is treated as the single -pattern to be matched when neither \fB-e\fP nor \fB-f\fP is present. -Conversely, when one or both of these options are used to specify patterns, all -arguments are treated as path names. At least one of \fB-e\fP, \fB-f\fP, or an -argument pattern must be provided. -.P -If no files are specified, \fBpcregrep\fP reads the standard input. The -standard input can also be referenced by a name consisting of a single hyphen. -For example: -.sp - pcregrep some-pattern /file1 - /file3 -.sp -By default, each line that matches a pattern is copied to the standard -output, and if there is more than one file, the file name is output at the -start of each line, followed by a colon. However, there are options that can -change how \fBpcregrep\fP behaves. In particular, the \fB-M\fP option makes it -possible to search for patterns that span line boundaries. What defines a line -boundary is controlled by the \fB-N\fP (\fB--newline\fP) option. -.P -The amount of memory used for buffering files that are being scanned is -controlled by a parameter that can be set by the \fB--buffer-size\fP option. -The default value for this parameter is specified when \fBpcregrep\fP is built, -with the default default being 20K. A block of memory three times this size is -used (to allow for buffering "before" and "after" lines). An error occurs if a -line overflows the buffer. -.P -Patterns can be no longer than 8K or BUFSIZ bytes, whichever is the greater. -BUFSIZ is defined in \fB<stdio.h>\fP. When there is more than one pattern -(specified by the use of \fB-e\fP and/or \fB-f\fP), each pattern is applied to -each line in the order in which they are defined, except that all the \fB-e\fP -patterns are tried before the \fB-f\fP patterns. -.P -By default, as soon as one pattern matches a line, no further patterns are -considered. However, if \fB--colour\fP (or \fB--color\fP) is used to colour the -matching substrings, or if \fB--only-matching\fP, \fB--file-offsets\fP, or -\fB--line-offsets\fP is used to output only the part of the line that matched -(either shown literally, or as an offset), scanning resumes immediately -following the match, so that further matches on the same line can be found. If -there are multiple patterns, they are all tried on the remainder of the line, -but patterns that follow the one that matched are not tried on the earlier part -of the line. -.P -This behaviour means that the order in which multiple patterns are specified -can affect the output when one of the above options is used. This is no longer -the same behaviour as GNU grep, which now manages to display earlier matches -for later patterns (as long as there is no overlap). -.P -Patterns that can match an empty string are accepted, but empty string -matches are never recognized. An example is the pattern "(super)?(man)?", in -which all components are optional. This pattern finds all occurrences of both -"super" and "man"; the output differs from matching with "super|man" when only -the matching substrings are being shown. -.P -If the \fBLC_ALL\fP or \fBLC_CTYPE\fP environment variable is set, -\fBpcregrep\fP uses the value to set a locale when calling the PCRE library. -The \fB--locale\fP option can be used to override this. -. -. -.SH "SUPPORT FOR COMPRESSED FILES" -.rs -.sp -It is possible to compile \fBpcregrep\fP so that it uses \fBlibz\fP or -\fBlibbz2\fP to read files whose names end in \fB.gz\fP or \fB.bz2\fP, -respectively. You can find out whether your binary has support for one or both -of these file types by running it with the \fB--help\fP option. If the -appropriate support is not present, files are treated as plain text. The -standard input is always so treated. -. -. -.SH "BINARY FILES" -.rs -.sp -By default, a file that contains a binary zero byte within the first 1024 bytes -is identified as a binary file, and is processed specially. (GNU grep also -identifies binary files in this manner.) See the \fB--binary-files\fP option -for a means of changing the way binary files are handled. -. -. -.SH OPTIONS -.rs -.sp -The order in which some of the options appear can affect the output. For -example, both the \fB-h\fP and \fB-l\fP options affect the printing of file -names. Whichever comes later in the command line will be the one that takes -effect. Similarly, except where noted below, if an option is given twice, the -later setting is used. Numerical values for options may be followed by K or M, -to signify multiplication by 1024 or 1024*1024 respectively. -.TP 10 -\fB--\fP -This terminates the list of options. It is useful if the next item on the -command line starts with a hyphen but is not an option. This allows for the -processing of patterns and filenames that start with hyphens. -.TP -\fB-A\fP \fInumber\fP, \fB--after-context=\fP\fInumber\fP -Output \fInumber\fP lines of context after each matching line. If filenames -and/or line numbers are being output, a hyphen separator is used instead of a -colon for the context lines. A line containing "--" is output between each -group of lines, unless they are in fact contiguous in the input file. The value -of \fInumber\fP is expected to be relatively small. However, \fBpcregrep\fP -guarantees to have up to 8K of following text available for context output. -.TP -\fB-a\fP, \fB--text\fP -Treat binary files as text. This is equivalent to -\fB--binary-files\fP=\fItext\fP. -.TP -\fB-B\fP \fInumber\fP, \fB--before-context=\fP\fInumber\fP -Output \fInumber\fP lines of context before each matching line. If filenames -and/or line numbers are being output, a hyphen separator is used instead of a -colon for the context lines. A line containing "--" is output between each -group of lines, unless they are in fact contiguous in the input file. The value -of \fInumber\fP is expected to be relatively small. However, \fBpcregrep\fP -guarantees to have up to 8K of preceding text available for context output. -.TP -\fB--binary-files=\fP\fIword\fP -Specify how binary files are to be processed. If the word is "binary" (the -default), pattern matching is performed on binary files, but the only output is -"Binary file <name> matches" when a match succeeds. If the word is "text", -which is equivalent to the \fB-a\fP or \fB--text\fP option, binary files are -processed in the same way as any other file. In this case, when a match -succeeds, the output may be binary garbage, which can have nasty effects if -sent to a terminal. If the word is "without-match", which is equivalent to the -\fB-I\fP option, binary files are not processed at all; they are assumed not to -be of interest. -.TP -\fB--buffer-size=\fP\fInumber\fP -Set the parameter that controls how much memory is used for buffering files -that are being scanned. -.TP -\fB-C\fP \fInumber\fP, \fB--context=\fP\fInumber\fP -Output \fInumber\fP lines of context both before and after each matching line. -This is equivalent to setting both \fB-A\fP and \fB-B\fP to the same value. -.TP -\fB-c\fP, \fB--count\fP -Do not output individual lines from the files that are being scanned; instead -output the number of lines that would otherwise have been shown. If no lines -are selected, the number zero is output. If several files are are being -scanned, a count is output for each of them. However, if the -\fB--files-with-matches\fP option is also used, only those files whose counts -are greater than zero are listed. When \fB-c\fP is used, the \fB-A\fP, -\fB-B\fP, and \fB-C\fP options are ignored. -.TP -\fB--colour\fP, \fB--color\fP -If this option is given without any data, it is equivalent to "--colour=auto". -If data is required, it must be given in the same shell item, separated by an -equals sign. -.TP -\fB--colour=\fP\fIvalue\fP, \fB--color=\fP\fIvalue\fP -This option specifies under what circumstances the parts of a line that matched -a pattern should be coloured in the output. By default, the output is not -coloured. The value (which is optional, see above) may be "never", "always", or -"auto". In the latter case, colouring happens only if the standard output is -connected to a terminal. More resources are used when colouring is enabled, -because \fBpcregrep\fP has to search for all possible matches in a line, not -just one, in order to colour them all. -.sp -The colour that is used can be specified by setting the environment variable -PCREGREP_COLOUR or PCREGREP_COLOR. The value of this variable should be a -string of two numbers, separated by a semicolon. They are copied directly into -the control string for setting colour on a terminal, so it is your -responsibility to ensure that they make sense. If neither of the environment -variables is set, the default is "1;31", which gives red. -.TP -\fB-D\fP \fIaction\fP, \fB--devices=\fP\fIaction\fP -If an input path is not a regular file or a directory, "action" specifies how -it is to be processed. Valid values are "read" (the default) or "skip" -(silently skip the path). -.TP -\fB-d\fP \fIaction\fP, \fB--directories=\fP\fIaction\fP -If an input path is a directory, "action" specifies how it is to be processed. -Valid values are "read" (the default in non-Windows environments, for -compatibility with GNU grep), "recurse" (equivalent to the \fB-r\fP option), or -"skip" (silently skip the path, the default in Windows environments). In the -"read" case, directories are read as if they were ordinary files. In some -operating systems the effect of reading a directory like this is an immediate -end-of-file; in others it may provoke an error. -.TP -\fB-e\fP \fIpattern\fP, \fB--regex=\fP\fIpattern\fP, \fB--regexp=\fP\fIpattern\fP -Specify a pattern to be matched. This option can be used multiple times in -order to specify several patterns. It can also be used as a way of specifying a -single pattern that starts with a hyphen. When \fB-e\fP is used, no argument -pattern is taken from the command line; all arguments are treated as file -names. There is no limit to the number of patterns. They are applied to each -line in the order in which they are defined until one matches. -.sp -If \fB-f\fP is used with \fB-e\fP, the command line patterns are matched first, -followed by the patterns from the file(s), independent of the order in which -these options are specified. Note that multiple use of \fB-e\fP is not the same -as a single pattern with alternatives. For example, X|Y finds the first -character in a line that is X or Y, whereas if the two patterns are given -separately, with X first, \fBpcregrep\fP finds X if it is present, even if it -follows Y in the line. It finds Y only if there is no X in the line. This -matters only if you are using \fB-o\fP or \fB--colo(u)r\fP to show the part(s) -of the line that matched. -.TP -\fB--exclude\fP=\fIpattern\fP -Files (but not directories) whose names match the pattern are skipped without -being processed. This applies to all files, whether listed on the command line, -obtained from \fB--file-list\fP, or by scanning a directory. The pattern is a -PCRE regular expression, and is matched against the final component of the file -name, not the entire path. The \fB-F\fP, \fB-w\fP, and \fB-x\fP options do not -apply to this pattern. The option may be given any number of times in order to -specify multiple patterns. If a file name matches both an \fB--include\fP -and an \fB--exclude\fP pattern, it is excluded. There is no short form for this -option. -.TP -\fB--exclude-from=\fP\fIfilename\fP -Treat each non-empty line of the file as the data for an \fB--exclude\fP -option. What constitutes a newline when reading the file is the operating -system's default. The \fB--newline\fP option has no effect on this option. This -option may be given more than once in order to specify a number of files to -read. -.TP -\fB--exclude-dir\fP=\fIpattern\fP -Directories whose names match the pattern are skipped without being processed, -whatever the setting of the \fB--recursive\fP option. This applies to all -directories, whether listed on the command line, obtained from -\fB--file-list\fP, or by scanning a parent directory. The pattern is a PCRE -regular expression, and is matched against the final component of the directory -name, not the entire path. The \fB-F\fP, \fB-w\fP, and \fB-x\fP options do not -apply to this pattern. The option may be given any number of times in order to -specify more than one pattern. If a directory matches both \fB--include-dir\fP -and \fB--exclude-dir\fP, it is excluded. There is no short form for this -option. -.TP -\fB-F\fP, \fB--fixed-strings\fP -Interpret each data-matching pattern as a list of fixed strings, separated by -newlines, instead of as a regular expression. What constitutes a newline for -this purpose is controlled by the \fB--newline\fP option. The \fB-w\fP (match -as a word) and \fB-x\fP (match whole line) options can be used with \fB-F\fP. -They apply to each of the fixed strings. A line is selected if any of the fixed -strings are found in it (subject to \fB-w\fP or \fB-x\fP, if present). This -option applies only to the patterns that are matched against the contents of -files; it does not apply to patterns specified by any of the \fB--include\fP or -\fB--exclude\fP options. -.TP -\fB-f\fP \fIfilename\fP, \fB--file=\fP\fIfilename\fP -Read patterns from the file, one per line, and match them against -each line of input. What constitutes a newline when reading the file is the -operating system's default. The \fB--newline\fP option has no effect on this -option. Trailing white space is removed from each line, and blank lines are -ignored. An empty file contains no patterns and therefore matches nothing. See -also the comments about multiple patterns versus a single pattern with -alternatives in the description of \fB-e\fP above. -.sp -If this option is given more than once, all the specified files are -read. A data line is output if any of the patterns match it. A filename can -be given as "-" to refer to the standard input. When \fB-f\fP is used, patterns -specified on the command line using \fB-e\fP may also be present; they are -tested before the file's patterns. However, no other pattern is taken from the -command line; all arguments are treated as the names of paths to be searched. -.TP -\fB--file-list\fP=\fIfilename\fP -Read a list of files and/or directories that are to be scanned from the given -file, one per line. Trailing white space is removed from each line, and blank -lines are ignored. These paths are processed before any that are listed on the -command line. The filename can be given as "-" to refer to the standard input. -If \fB--file\fP and \fB--file-list\fP are both specified as "-", patterns are -read first. This is useful only when the standard input is a terminal, from -which further lines (the list of files) can be read after an end-of-file -indication. If this option is given more than once, all the specified files are -read. -.TP -\fB--file-offsets\fP -Instead of showing lines or parts of lines that match, show each match as an -offset from the start of the file and a length, separated by a comma. In this -mode, no context is shown. That is, the \fB-A\fP, \fB-B\fP, and \fB-C\fP -options are ignored. If there is more than one match in a line, each of them is -shown separately. This option is mutually exclusive with \fB--line-offsets\fP -and \fB--only-matching\fP. -.TP -\fB-H\fP, \fB--with-filename\fP -Force the inclusion of the filename at the start of output lines when searching -a single file. By default, the filename is not shown in this case. For matching -lines, the filename is followed by a colon; for context lines, a hyphen -separator is used. If a line number is also being output, it follows the file -name. -.TP -\fB-h\fP, \fB--no-filename\fP -Suppress the output filenames when searching multiple files. By default, -filenames are shown when multiple files are searched. For matching lines, the -filename is followed by a colon; for context lines, a hyphen separator is used. -If a line number is also being output, it follows the file name. -.TP -\fB--help\fP -Output a help message, giving brief details of the command options and file -type support, and then exit. Anything else on the command line is -ignored. -.TP -\fB-I\fP -Treat binary files as never matching. This is equivalent to -\fB--binary-files\fP=\fIwithout-match\fP. -.TP -\fB-i\fP, \fB--ignore-case\fP -Ignore upper/lower case distinctions during comparisons. -.TP -\fB--include\fP=\fIpattern\fP -If any \fB--include\fP patterns are specified, the only files that are -processed are those that match one of the patterns (and do not match an -\fB--exclude\fP pattern). This option does not affect directories, but it -applies to all files, whether listed on the command line, obtained from -\fB--file-list\fP, or by scanning a directory. The pattern is a PCRE regular -expression, and is matched against the final component of the file name, not -the entire path. The \fB-F\fP, \fB-w\fP, and \fB-x\fP options do not apply to -this pattern. The option may be given any number of times. If a file name -matches both an \fB--include\fP and an \fB--exclude\fP pattern, it is excluded. -There is no short form for this option. -.TP -\fB--include-from=\fP\fIfilename\fP -Treat each non-empty line of the file as the data for an \fB--include\fP -option. What constitutes a newline for this purpose is the operating system's -default. The \fB--newline\fP option has no effect on this option. This option -may be given any number of times; all the files are read. -.TP -\fB--include-dir\fP=\fIpattern\fP -If any \fB--include-dir\fP patterns are specified, the only directories that -are processed are those that match one of the patterns (and do not match an -\fB--exclude-dir\fP pattern). This applies to all directories, whether listed -on the command line, obtained from \fB--file-list\fP, or by scanning a parent -directory. The pattern is a PCRE regular expression, and is matched against the -final component of the directory name, not the entire path. The \fB-F\fP, -\fB-w\fP, and \fB-x\fP options do not apply to this pattern. The option may be -given any number of times. If a directory matches both \fB--include-dir\fP and -\fB--exclude-dir\fP, it is excluded. There is no short form for this option. -.TP -\fB-L\fP, \fB--files-without-match\fP -Instead of outputting lines from the files, just output the names of the files -that do not contain any lines that would have been output. Each file name is -output once, on a separate line. -.TP -\fB-l\fP, \fB--files-with-matches\fP -Instead of outputting lines from the files, just output the names of the files -containing lines that would have been output. Each file name is output -once, on a separate line. Searching normally stops as soon as a matching line -is found in a file. However, if the \fB-c\fP (count) option is also used, -matching continues in order to obtain the correct count, and those files that -have at least one match are listed along with their counts. Using this option -with \fB-c\fP is a way of suppressing the listing of files with no matches. -.TP -\fB--label\fP=\fIname\fP -This option supplies a name to be used for the standard input when file names -are being output. If not supplied, "(standard input)" is used. There is no -short form for this option. -.TP -\fB--line-buffered\fP -When this option is given, input is read and processed line by line, and the -output is flushed after each write. By default, input is read in large chunks, -unless \fBpcregrep\fP can determine that it is reading from a terminal (which -is currently possible only in Unix-like environments). Output to terminal is -normally automatically flushed by the operating system. This option can be -useful when the input or output is attached to a pipe and you do not want -\fBpcregrep\fP to buffer up large amounts of data. However, its use will affect -performance, and the \fB-M\fP (multiline) option ceases to work. -.TP -\fB--line-offsets\fP -Instead of showing lines or parts of lines that match, show each match as a -line number, the offset from the start of the line, and a length. The line -number is terminated by a colon (as usual; see the \fB-n\fP option), and the -offset and length are separated by a comma. In this mode, no context is shown. -That is, the \fB-A\fP, \fB-B\fP, and \fB-C\fP options are ignored. If there is -more than one match in a line, each of them is shown separately. This option is -mutually exclusive with \fB--file-offsets\fP and \fB--only-matching\fP. -.TP -\fB--locale\fP=\fIlocale-name\fP -This option specifies a locale to be used for pattern matching. It overrides -the value in the \fBLC_ALL\fP or \fBLC_CTYPE\fP environment variables. If no -locale is specified, the PCRE library's default (usually the "C" locale) is -used. There is no short form for this option. -.TP -\fB--match-limit\fP=\fInumber\fP -Processing some regular expression patterns can require a very large amount of -memory, leading in some cases to a program crash if not enough is available. -Other patterns may take a very long time to search for all possible matching -strings. The \fBpcre_exec()\fP function that is called by \fBpcregrep\fP to do -the matching has two parameters that can limit the resources that it uses. -.sp -The \fB--match-limit\fP option provides a means of limiting resource usage -when processing patterns that are not going to match, but which have a very -large number of possibilities in their search trees. The classic example is a -pattern that uses nested unlimited repeats. Internally, PCRE uses a function -called \fBmatch()\fP which it calls repeatedly (sometimes recursively). The -limit set by \fB--match-limit\fP is imposed on the number of times this -function is called during a match, which has the effect of limiting the amount -of backtracking that can take place. -.sp -The \fB--recursion-limit\fP option is similar to \fB--match-limit\fP, but -instead of limiting the total number of times that \fBmatch()\fP is called, it -limits the depth of recursive calls, which in turn limits the amount of memory -that can be used. The recursion depth is a smaller number than the total number -of calls, because not all calls to \fBmatch()\fP are recursive. This limit is -of use only if it is set smaller than \fB--match-limit\fP. -.sp -There are no short forms for these options. The default settings are specified -when the PCRE library is compiled, with the default default being 10 million. -.TP -\fB-M\fP, \fB--multiline\fP -Allow patterns to match more than one line. When this option is given, patterns -may usefully contain literal newline characters and internal occurrences of ^ -and $ characters. The output for a successful match may consist of more than -one line, the last of which is the one in which the match ended. If the matched -string ends with a newline sequence the output ends at the end of that line. -.sp -When this option is set, the PCRE library is called in "multiline" mode. -There is a limit to the number of lines that can be matched, imposed by the way -that \fBpcregrep\fP buffers the input file as it scans it. However, -\fBpcregrep\fP ensures that at least 8K characters or the rest of the document -(whichever is the shorter) are available for forward matching, and similarly -the previous 8K characters (or all the previous characters, if fewer than 8K) -are guaranteed to be available for lookbehind assertions. This option does not -work when input is read line by line (see \fP--line-buffered\fP.) -.TP -\fB-N\fP \fInewline-type\fP, \fB--newline\fP=\fInewline-type\fP -The PCRE library supports five different conventions for indicating -the ends of lines. They are the single-character sequences CR (carriage return) -and LF (linefeed), the two-character sequence CRLF, an "anycrlf" convention, -which recognizes any of the preceding three types, and an "any" convention, in -which any Unicode line ending sequence is assumed to end a line. The Unicode -sequences are the three just mentioned, plus VT (vertical tab, U+000B), FF -(form feed, U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and -PS (paragraph separator, U+2029). -.sp -When the PCRE library is built, a default line-ending sequence is specified. -This is normally the standard sequence for the operating system. Unless -otherwise specified by this option, \fBpcregrep\fP uses the library's default. -The possible values for this option are CR, LF, CRLF, ANYCRLF, or ANY. This -makes it possible to use \fBpcregrep\fP to scan files that have come from other -environments without having to modify their line endings. If the data that is -being scanned does not agree with the convention set by this option, -\fBpcregrep\fP may behave in strange ways. Note that this option does not -apply to files specified by the \fB-f\fP, \fB--exclude-from\fP, or -\fB--include-from\fP options, which are expected to use the operating system's -standard newline sequence. -.TP -\fB-n\fP, \fB--line-number\fP -Precede each output line by its line number in the file, followed by a colon -for matching lines or a hyphen for context lines. If the filename is also being -output, it precedes the line number. This option is forced if -\fB--line-offsets\fP is used. -.TP -\fB--no-jit\fP -If the PCRE library is built with support for just-in-time compiling (which -speeds up matching), \fBpcregrep\fP automatically makes use of this, unless it -was explicitly disabled at build time. This option can be used to disable the -use of JIT at run time. It is provided for testing and working round problems. -It should never be needed in normal use. -.TP -\fB-o\fP, \fB--only-matching\fP -Show only the part of the line that matched a pattern instead of the whole -line. In this mode, no context is shown. That is, the \fB-A\fP, \fB-B\fP, and -\fB-C\fP options are ignored. If there is more than one match in a line, each -of them is shown separately. If \fB-o\fP is combined with \fB-v\fP (invert the -sense of the match to find non-matching lines), no output is generated, but the -return code is set appropriately. If the matched portion of the line is empty, -nothing is output unless the file name or line number are being printed, in -which case they are shown on an otherwise empty line. This option is mutually -exclusive with \fB--file-offsets\fP and \fB--line-offsets\fP. -.TP -\fB-o\fP\fInumber\fP, \fB--only-matching\fP=\fInumber\fP -Show only the part of the line that matched the capturing parentheses of the -given number. Up to 32 capturing parentheses are supported, and -o0 is -equivalent to \fB-o\fP without a number. Because these options can be given -without an argument (see above), if an argument is present, it must be given in -the same shell item, for example, -o3 or --only-matching=2. The comments given -for the non-argument case above also apply to this case. If the specified -capturing parentheses do not exist in the pattern, or were not set in the -match, nothing is output unless the file name or line number are being printed. -.sp -If this option is given multiple times, multiple substrings are output, in the -order the options are given. For example, -o3 -o1 -o3 causes the substrings -matched by capturing parentheses 3 and 1 and then 3 again to be output. By -default, there is no separator (but see the next option). -.TP -\fB--om-separator\fP=\fItext\fP -Specify a separating string for multiple occurrences of \fB-o\fP. The default -is an empty string. Separating strings are never coloured. -.TP -\fB-q\fP, \fB--quiet\fP -Work quietly, that is, display nothing except error messages. The exit -status indicates whether or not any matches were found. -.TP -\fB-r\fP, \fB--recursive\fP -If any given path is a directory, recursively scan the files it contains, -taking note of any \fB--include\fP and \fB--exclude\fP settings. By default, a -directory is read as a normal file; in some operating systems this gives an -immediate end-of-file. This option is a shorthand for setting the \fB-d\fP -option to "recurse". -.TP -\fB--recursion-limit\fP=\fInumber\fP -See \fB--match-limit\fP above. -.TP -\fB-s\fP, \fB--no-messages\fP -Suppress error messages about non-existent or unreadable files. Such files are -quietly skipped. However, the return code is still 2, even if matches were -found in other files. -.TP -\fB-u\fP, \fB--utf-8\fP -Operate in UTF-8 mode. This option is available only if PCRE has been compiled -with UTF-8 support. All patterns (including those for any \fB--exclude\fP and -\fB--include\fP options) and all subject lines that are scanned must be valid -strings of UTF-8 characters. -.TP -\fB-V\fP, \fB--version\fP -Write the version numbers of \fBpcregrep\fP and the PCRE library to the -standard output and then exit. Anything else on the command line is -ignored. -.TP -\fB-v\fP, \fB--invert-match\fP -Invert the sense of the match, so that lines which do \fInot\fP match any of -the patterns are the ones that are found. -.TP -\fB-w\fP, \fB--word-regex\fP, \fB--word-regexp\fP -Force the patterns to match only whole words. This is equivalent to having \eb -at the start and end of the pattern. This option applies only to the patterns -that are matched against the contents of files; it does not apply to patterns -specified by any of the \fB--include\fP or \fB--exclude\fP options. -.TP -\fB-x\fP, \fB--line-regex\fP, \fB--line-regexp\fP -Force the patterns to be anchored (each must start matching at the beginning of -a line) and in addition, require them to match entire lines. This is equivalent -to having ^ and $ characters at the start and end of each alternative branch in -every pattern. This option applies only to the patterns that are matched -against the contents of files; it does not apply to patterns specified by any -of the \fB--include\fP or \fB--exclude\fP options. -. -. -.SH "ENVIRONMENT VARIABLES" -.rs -.sp -The environment variables \fBLC_ALL\fP and \fBLC_CTYPE\fP are examined, in that -order, for a locale. The first one that is set is used. This can be overridden -by the \fB--locale\fP option. If no locale is set, the PCRE library's default -(usually the "C" locale) is used. -. -. -.SH "NEWLINES" -.rs -.sp -The \fB-N\fP (\fB--newline\fP) option allows \fBpcregrep\fP to scan files with -different newline conventions from the default. Any parts of the input files -that are written to the standard output are copied identically, with whatever -newline sequences they have in the input. However, the setting of this option -does not affect the interpretation of files specified by the \fB-f\fP, -\fB--exclude-from\fP, or \fB--include-from\fP options, which are assumed to use -the operating system's standard newline sequence, nor does it affect the way in -which \fBpcregrep\fP writes informational messages to the standard error and -output streams. For these it uses the string "\en" to indicate newlines, -relying on the C I/O library to convert this to an appropriate sequence. -. -. -.SH "OPTIONS COMPATIBILITY" -.rs -.sp -Many of the short and long forms of \fBpcregrep\fP's options are the same -as in the GNU \fBgrep\fP program. Any long option of the form -\fB--xxx-regexp\fP (GNU terminology) is also available as \fB--xxx-regex\fP -(PCRE terminology). However, the \fB--file-list\fP, \fB--file-offsets\fP, -\fB--include-dir\fP, \fB--line-offsets\fP, \fB--locale\fP, \fB--match-limit\fP, -\fB-M\fP, \fB--multiline\fP, \fB-N\fP, \fB--newline\fP, \fB--om-separator\fP, -\fB--recursion-limit\fP, \fB-u\fP, and \fB--utf-8\fP options are specific to -\fBpcregrep\fP, as is the use of the \fB--only-matching\fP option with a -capturing parentheses number. -.P -Although most of the common options work the same way, a few are different in -\fBpcregrep\fP. For example, the \fB--include\fP option's argument is a glob -for GNU \fBgrep\fP, but a regular expression for \fBpcregrep\fP. If both the -\fB-c\fP and \fB-l\fP options are given, GNU grep lists only file names, -without counts, but \fBpcregrep\fP gives the counts. -. -. -.SH "OPTIONS WITH DATA" -.rs -.sp -There are four different ways in which an option with data can be specified. -If a short form option is used, the data may follow immediately, or (with one -exception) in the next command line item. For example: -.sp - -f/some/file - -f /some/file -.sp -The exception is the \fB-o\fP option, which may appear with or without data. -Because of this, if data is present, it must follow immediately in the same -item, for example -o3. -.P -If a long form option is used, the data may appear in the same command line -item, separated by an equals character, or (with two exceptions) it may appear -in the next command line item. For example: -.sp - --file=/some/file - --file /some/file -.sp -Note, however, that if you want to supply a file name beginning with ~ as data -in a shell command, and have the shell expand ~ to a home directory, you must -separate the file name from the option, because the shell does not treat ~ -specially unless it is at the start of an item. -.P -The exceptions to the above are the \fB--colour\fP (or \fB--color\fP) and -\fB--only-matching\fP options, for which the data is optional. If one of these -options does have data, it must be given in the first form, using an equals -character. Otherwise \fBpcregrep\fP will assume that it has no data. -. -. -.SH "MATCHING ERRORS" -.rs -.sp -It is possible to supply a regular expression that takes a very long time to -fail to match certain lines. Such patterns normally involve nested indefinite -repeats, for example: (a+)*\ed when matched against a line of a's with no final -digit. The PCRE matching function has a resource limit that causes it to abort -in these circumstances. If this happens, \fBpcregrep\fP outputs an error -message and the line that caused the problem to the standard error stream. If -there are more than 20 such errors, \fBpcregrep\fP gives up. -.P -The \fB--match-limit\fP option of \fBpcregrep\fP can be used to set the overall -resource limit; there is a second option called \fB--recursion-limit\fP that -sets a limit on the amount of memory (usually stack) that is used (see the -discussion of these options above). -. -. -.SH DIAGNOSTICS -.rs -.sp -Exit status is 0 if any matches were found, 1 if no matches were found, and 2 -for syntax errors, overlong lines, non-existent or inaccessible files (even if -matches were found in other files) or too many matching errors. Using the -\fB-s\fP option to suppress error messages about inaccessible files does not -affect the return code. -. -. -.SH "SEE ALSO" -.rs -.sp -\fBpcrepattern\fP(3), \fBpcresyntax\fP(3), \fBpcretest\fP(1). -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 03 April 2014 -Copyright (c) 1997-2014 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcregrep.txt b/plugins/Pcre16/docs/doc/pcregrep.txt deleted file mode 100644 index 97d9a7bd37..0000000000 --- a/plugins/Pcre16/docs/doc/pcregrep.txt +++ /dev/null @@ -1,741 +0,0 @@ -PCREGREP(1) General Commands Manual PCREGREP(1) - - - -NAME - pcregrep - a grep with Perl-compatible regular expressions. - -SYNOPSIS - pcregrep [options] [long options] [pattern] [path1 path2 ...] - - -DESCRIPTION - - pcregrep searches files for character patterns, in the same way as - other grep commands do, but it uses the PCRE regular expression library - to support patterns that are compatible with the regular expressions of - Perl 5. See pcresyntax(3) for a quick-reference summary of pattern syn- - tax, or pcrepattern(3) for a full description of the syntax and seman- - tics of the regular expressions that PCRE supports. - - Patterns, whether supplied on the command line or in a separate file, - are given without delimiters. For example: - - pcregrep Thursday /etc/motd - - If you attempt to use delimiters (for example, by surrounding a pattern - with slashes, as is common in Perl scripts), they are interpreted as - part of the pattern. Quotes can of course be used to delimit patterns - on the command line because they are interpreted by the shell, and - indeed quotes are required if a pattern contains white space or shell - metacharacters. - - The first argument that follows any option settings is treated as the - single pattern to be matched when neither -e nor -f is present. Con- - versely, when one or both of these options are used to specify pat- - terns, all arguments are treated as path names. At least one of -e, -f, - or an argument pattern must be provided. - - If no files are specified, pcregrep reads the standard input. The stan- - dard input can also be referenced by a name consisting of a single - hyphen. For example: - - pcregrep some-pattern /file1 - /file3 - - By default, each line that matches a pattern is copied to the standard - output, and if there is more than one file, the file name is output at - the start of each line, followed by a colon. However, there are options - that can change how pcregrep behaves. In particular, the -M option - makes it possible to search for patterns that span line boundaries. - What defines a line boundary is controlled by the -N (--newline) - option. - - The amount of memory used for buffering files that are being scanned is - controlled by a parameter that can be set by the --buffer-size option. - The default value for this parameter is specified when pcregrep is - built, with the default default being 20K. A block of memory three - times this size is used (to allow for buffering "before" and "after" - lines). An error occurs if a line overflows the buffer. - - Patterns can be no longer than 8K or BUFSIZ bytes, whichever is the - greater. BUFSIZ is defined in <stdio.h>. When there is more than one - pattern (specified by the use of -e and/or -f), each pattern is applied - to each line in the order in which they are defined, except that all - the -e patterns are tried before the -f patterns. - - By default, as soon as one pattern matches a line, no further patterns - are considered. However, if --colour (or --color) is used to colour the - matching substrings, or if --only-matching, --file-offsets, or --line- - offsets is used to output only the part of the line that matched - (either shown literally, or as an offset), scanning resumes immediately - following the match, so that further matches on the same line can be - found. If there are multiple patterns, they are all tried on the - remainder of the line, but patterns that follow the one that matched - are not tried on the earlier part of the line. - - This behaviour means that the order in which multiple patterns are - specified can affect the output when one of the above options is used. - This is no longer the same behaviour as GNU grep, which now manages to - display earlier matches for later patterns (as long as there is no - overlap). - - Patterns that can match an empty string are accepted, but empty string - matches are never recognized. An example is the pattern - "(super)?(man)?", in which all components are optional. This pattern - finds all occurrences of both "super" and "man"; the output differs - from matching with "super|man" when only the matching substrings are - being shown. - - If the LC_ALL or LC_CTYPE environment variable is set, pcregrep uses - the value to set a locale when calling the PCRE library. The --locale - option can be used to override this. - - -SUPPORT FOR COMPRESSED FILES - - It is possible to compile pcregrep so that it uses libz or libbz2 to - read files whose names end in .gz or .bz2, respectively. You can find - out whether your binary has support for one or both of these file types - by running it with the --help option. If the appropriate support is not - present, files are treated as plain text. The standard input is always - so treated. - - -BINARY FILES - - By default, a file that contains a binary zero byte within the first - 1024 bytes is identified as a binary file, and is processed specially. - (GNU grep also identifies binary files in this manner.) See the - --binary-files option for a means of changing the way binary files are - handled. - - -OPTIONS - - The order in which some of the options appear can affect the output. - For example, both the -h and -l options affect the printing of file - names. Whichever comes later in the command line will be the one that - takes effect. Similarly, except where noted below, if an option is - given twice, the later setting is used. Numerical values for options - may be followed by K or M, to signify multiplication by 1024 or - 1024*1024 respectively. - - -- This terminates the list of options. It is useful if the next - item on the command line starts with a hyphen but is not an - option. This allows for the processing of patterns and file- - names that start with hyphens. - - -A number, --after-context=number - Output number lines of context after each matching line. If - filenames and/or line numbers are being output, a hyphen sep- - arator is used instead of a colon for the context lines. A - line containing "--" is output between each group of lines, - unless they are in fact contiguous in the input file. The - value of number is expected to be relatively small. However, - pcregrep guarantees to have up to 8K of following text avail- - able for context output. - - -a, --text - Treat binary files as text. This is equivalent to --binary- - files=text. - - -B number, --before-context=number - Output number lines of context before each matching line. If - filenames and/or line numbers are being output, a hyphen sep- - arator is used instead of a colon for the context lines. A - line containing "--" is output between each group of lines, - unless they are in fact contiguous in the input file. The - value of number is expected to be relatively small. However, - pcregrep guarantees to have up to 8K of preceding text avail- - able for context output. - - --binary-files=word - Specify how binary files are to be processed. If the word is - "binary" (the default), pattern matching is performed on - binary files, but the only output is "Binary file <name> - matches" when a match succeeds. If the word is "text", which - is equivalent to the -a or --text option, binary files are - processed in the same way as any other file. In this case, - when a match succeeds, the output may be binary garbage, - which can have nasty effects if sent to a terminal. If the - word is "without-match", which is equivalent to the -I - option, binary files are not processed at all; they are - assumed not to be of interest. - - --buffer-size=number - Set the parameter that controls how much memory is used for - buffering files that are being scanned. - - -C number, --context=number - Output number lines of context both before and after each - matching line. This is equivalent to setting both -A and -B - to the same value. - - -c, --count - Do not output individual lines from the files that are being - scanned; instead output the number of lines that would other- - wise have been shown. If no lines are selected, the number - zero is output. If several files are are being scanned, a - count is output for each of them. However, if the --files- - with-matches option is also used, only those files whose - counts are greater than zero are listed. When -c is used, the - -A, -B, and -C options are ignored. - - --colour, --color - If this option is given without any data, it is equivalent to - "--colour=auto". If data is required, it must be given in - the same shell item, separated by an equals sign. - - --colour=value, --color=value - This option specifies under what circumstances the parts of a - line that matched a pattern should be coloured in the output. - By default, the output is not coloured. The value (which is - optional, see above) may be "never", "always", or "auto". In - the latter case, colouring happens only if the standard out- - put is connected to a terminal. More resources are used when - colouring is enabled, because pcregrep has to search for all - possible matches in a line, not just one, in order to colour - them all. - - The colour that is used can be specified by setting the envi- - ronment variable PCREGREP_COLOUR or PCREGREP_COLOR. The value - of this variable should be a string of two numbers, separated - by a semicolon. They are copied directly into the control - string for setting colour on a terminal, so it is your - responsibility to ensure that they make sense. If neither of - the environment variables is set, the default is "1;31", - which gives red. - - -D action, --devices=action - If an input path is not a regular file or a directory, - "action" specifies how it is to be processed. Valid values - are "read" (the default) or "skip" (silently skip the path). - - -d action, --directories=action - If an input path is a directory, "action" specifies how it is - to be processed. Valid values are "read" (the default in - non-Windows environments, for compatibility with GNU grep), - "recurse" (equivalent to the -r option), or "skip" (silently - skip the path, the default in Windows environments). In the - "read" case, directories are read as if they were ordinary - files. In some operating systems the effect of reading a - directory like this is an immediate end-of-file; in others it - may provoke an error. - - -e pattern, --regex=pattern, --regexp=pattern - Specify a pattern to be matched. This option can be used mul- - tiple times in order to specify several patterns. It can also - be used as a way of specifying a single pattern that starts - with a hyphen. When -e is used, no argument pattern is taken - from the command line; all arguments are treated as file - names. There is no limit to the number of patterns. They are - applied to each line in the order in which they are defined - until one matches. - - If -f is used with -e, the command line patterns are matched - first, followed by the patterns from the file(s), independent - of the order in which these options are specified. Note that - multiple use of -e is not the same as a single pattern with - alternatives. For example, X|Y finds the first character in a - line that is X or Y, whereas if the two patterns are given - separately, with X first, pcregrep finds X if it is present, - even if it follows Y in the line. It finds Y only if there is - no X in the line. This matters only if you are using -o or - --colo(u)r to show the part(s) of the line that matched. - - --exclude=pattern - Files (but not directories) whose names match the pattern are - skipped without being processed. This applies to all files, - whether listed on the command line, obtained from --file- - list, or by scanning a directory. The pattern is a PCRE regu- - lar expression, and is matched against the final component of - the file name, not the entire path. The -F, -w, and -x - options do not apply to this pattern. The option may be given - any number of times in order to specify multiple patterns. If - a file name matches both an --include and an --exclude pat- - tern, it is excluded. There is no short form for this option. - - --exclude-from=filename - Treat each non-empty line of the file as the data for an - --exclude option. What constitutes a newline when reading the - file is the operating system's default. The --newline option - has no effect on this option. This option may be given more - than once in order to specify a number of files to read. - - --exclude-dir=pattern - Directories whose names match the pattern are skipped without - being processed, whatever the setting of the --recursive - option. This applies to all directories, whether listed on - the command line, obtained from --file-list, or by scanning a - parent directory. The pattern is a PCRE regular expression, - and is matched against the final component of the directory - name, not the entire path. The -F, -w, and -x options do not - apply to this pattern. The option may be given any number of - times in order to specify more than one pattern. If a direc- - tory matches both --include-dir and --exclude-dir, it is - excluded. There is no short form for this option. - - -F, --fixed-strings - Interpret each data-matching pattern as a list of fixed - strings, separated by newlines, instead of as a regular - expression. What constitutes a newline for this purpose is - controlled by the --newline option. The -w (match as a word) - and -x (match whole line) options can be used with -F. They - apply to each of the fixed strings. A line is selected if any - of the fixed strings are found in it (subject to -w or -x, if - present). This option applies only to the patterns that are - matched against the contents of files; it does not apply to - patterns specified by any of the --include or --exclude - options. - - -f filename, --file=filename - Read patterns from the file, one per line, and match them - against each line of input. What constitutes a newline when - reading the file is the operating system's default. The - --newline option has no effect on this option. Trailing white - space is removed from each line, and blank lines are ignored. - An empty file contains no patterns and therefore matches - nothing. See also the comments about multiple patterns versus - a single pattern with alternatives in the description of -e - above. - - If this option is given more than once, all the specified - files are read. A data line is output if any of the patterns - match it. A filename can be given as "-" to refer to the - standard input. When -f is used, patterns specified on the - command line using -e may also be present; they are tested - before the file's patterns. However, no other pattern is - taken from the command line; all arguments are treated as the - names of paths to be searched. - - --file-list=filename - Read a list of files and/or directories that are to be - scanned from the given file, one per line. Trailing white - space is removed from each line, and blank lines are ignored. - These paths are processed before any that are listed on the - command line. The filename can be given as "-" to refer to - the standard input. If --file and --file-list are both spec- - ified as "-", patterns are read first. This is useful only - when the standard input is a terminal, from which further - lines (the list of files) can be read after an end-of-file - indication. If this option is given more than once, all the - specified files are read. - - --file-offsets - Instead of showing lines or parts of lines that match, show - each match as an offset from the start of the file and a - length, separated by a comma. In this mode, no context is - shown. That is, the -A, -B, and -C options are ignored. If - there is more than one match in a line, each of them is shown - separately. This option is mutually exclusive with --line- - offsets and --only-matching. - - -H, --with-filename - Force the inclusion of the filename at the start of output - lines when searching a single file. By default, the filename - is not shown in this case. For matching lines, the filename - is followed by a colon; for context lines, a hyphen separator - is used. If a line number is also being output, it follows - the file name. - - -h, --no-filename - Suppress the output filenames when searching multiple files. - By default, filenames are shown when multiple files are - searched. For matching lines, the filename is followed by a - colon; for context lines, a hyphen separator is used. If a - line number is also being output, it follows the file name. - - --help Output a help message, giving brief details of the command - options and file type support, and then exit. Anything else - on the command line is ignored. - - -I Treat binary files as never matching. This is equivalent to - --binary-files=without-match. - - -i, --ignore-case - Ignore upper/lower case distinctions during comparisons. - - --include=pattern - If any --include patterns are specified, the only files that - are processed are those that match one of the patterns (and - do not match an --exclude pattern). This option does not - affect directories, but it applies to all files, whether - listed on the command line, obtained from --file-list, or by - scanning a directory. The pattern is a PCRE regular expres- - sion, and is matched against the final component of the file - name, not the entire path. The -F, -w, and -x options do not - apply to this pattern. The option may be given any number of - times. If a file name matches both an --include and an - --exclude pattern, it is excluded. There is no short form - for this option. - - --include-from=filename - Treat each non-empty line of the file as the data for an - --include option. What constitutes a newline for this purpose - is the operating system's default. The --newline option has - no effect on this option. This option may be given any number - of times; all the files are read. - - --include-dir=pattern - If any --include-dir patterns are specified, the only direc- - tories that are processed are those that match one of the - patterns (and do not match an --exclude-dir pattern). This - applies to all directories, whether listed on the command - line, obtained from --file-list, or by scanning a parent - directory. The pattern is a PCRE regular expression, and is - matched against the final component of the directory name, - not the entire path. The -F, -w, and -x options do not apply - to this pattern. The option may be given any number of times. - If a directory matches both --include-dir and --exclude-dir, - it is excluded. There is no short form for this option. - - -L, --files-without-match - Instead of outputting lines from the files, just output the - names of the files that do not contain any lines that would - have been output. Each file name is output once, on a sepa- - rate line. - - -l, --files-with-matches - Instead of outputting lines from the files, just output the - names of the files containing lines that would have been out- - put. Each file name is output once, on a separate line. - Searching normally stops as soon as a matching line is found - in a file. However, if the -c (count) option is also used, - matching continues in order to obtain the correct count, and - those files that have at least one match are listed along - with their counts. Using this option with -c is a way of sup- - pressing the listing of files with no matches. - - --label=name - This option supplies a name to be used for the standard input - when file names are being output. If not supplied, "(standard - input)" is used. There is no short form for this option. - - --line-buffered - When this option is given, input is read and processed line - by line, and the output is flushed after each write. By - default, input is read in large chunks, unless pcregrep can - determine that it is reading from a terminal (which is cur- - rently possible only in Unix-like environments). Output to - terminal is normally automatically flushed by the operating - system. This option can be useful when the input or output is - attached to a pipe and you do not want pcregrep to buffer up - large amounts of data. However, its use will affect perfor- - mance, and the -M (multiline) option ceases to work. - - --line-offsets - Instead of showing lines or parts of lines that match, show - each match as a line number, the offset from the start of the - line, and a length. The line number is terminated by a colon - (as usual; see the -n option), and the offset and length are - separated by a comma. In this mode, no context is shown. - That is, the -A, -B, and -C options are ignored. If there is - more than one match in a line, each of them is shown sepa- - rately. This option is mutually exclusive with --file-offsets - and --only-matching. - - --locale=locale-name - This option specifies a locale to be used for pattern match- - ing. It overrides the value in the LC_ALL or LC_CTYPE envi- - ronment variables. If no locale is specified, the PCRE - library's default (usually the "C" locale) is used. There is - no short form for this option. - - --match-limit=number - Processing some regular expression patterns can require a - very large amount of memory, leading in some cases to a pro- - gram crash if not enough is available. Other patterns may - take a very long time to search for all possible matching - strings. The pcre_exec() function that is called by pcregrep - to do the matching has two parameters that can limit the - resources that it uses. - - The --match-limit option provides a means of limiting - resource usage when processing patterns that are not going to - match, but which have a very large number of possibilities in - their search trees. The classic example is a pattern that - uses nested unlimited repeats. Internally, PCRE uses a func- - tion called match() which it calls repeatedly (sometimes - recursively). The limit set by --match-limit is imposed on - the number of times this function is called during a match, - which has the effect of limiting the amount of backtracking - that can take place. - - The --recursion-limit option is similar to --match-limit, but - instead of limiting the total number of times that match() is - called, it limits the depth of recursive calls, which in turn - limits the amount of memory that can be used. The recursion - depth is a smaller number than the total number of calls, - because not all calls to match() are recursive. This limit is - of use only if it is set smaller than --match-limit. - - There are no short forms for these options. The default set- - tings are specified when the PCRE library is compiled, with - the default default being 10 million. - - -M, --multiline - Allow patterns to match more than one line. When this option - is given, patterns may usefully contain literal newline char- - acters and internal occurrences of ^ and $ characters. The - output for a successful match may consist of more than one - line, the last of which is the one in which the match ended. - If the matched string ends with a newline sequence the output - ends at the end of that line. - - When this option is set, the PCRE library is called in "mul- - tiline" mode. There is a limit to the number of lines that - can be matched, imposed by the way that pcregrep buffers the - input file as it scans it. However, pcregrep ensures that at - least 8K characters or the rest of the document (whichever is - the shorter) are available for forward matching, and simi- - larly the previous 8K characters (or all the previous charac- - ters, if fewer than 8K) are guaranteed to be available for - lookbehind assertions. This option does not work when input - is read line by line (see --line-buffered.) - - -N newline-type, --newline=newline-type - The PCRE library supports five different conventions for - indicating the ends of lines. They are the single-character - sequences CR (carriage return) and LF (linefeed), the two- - character sequence CRLF, an "anycrlf" convention, which rec- - ognizes any of the preceding three types, and an "any" con- - vention, in which any Unicode line ending sequence is assumed - to end a line. The Unicode sequences are the three just men- - tioned, plus VT (vertical tab, U+000B), FF (form feed, - U+000C), NEL (next line, U+0085), LS (line separator, - U+2028), and PS (paragraph separator, U+2029). - - When the PCRE library is built, a default line-ending - sequence is specified. This is normally the standard - sequence for the operating system. Unless otherwise specified - by this option, pcregrep uses the library's default. The - possible values for this option are CR, LF, CRLF, ANYCRLF, or - ANY. This makes it possible to use pcregrep to scan files - that have come from other environments without having to mod- - ify their line endings. If the data that is being scanned - does not agree with the convention set by this option, pcre- - grep may behave in strange ways. Note that this option does - not apply to files specified by the -f, --exclude-from, or - --include-from options, which are expected to use the operat- - ing system's standard newline sequence. - - -n, --line-number - Precede each output line by its line number in the file, fol- - lowed by a colon for matching lines or a hyphen for context - lines. If the filename is also being output, it precedes the - line number. This option is forced if --line-offsets is used. - - --no-jit If the PCRE library is built with support for just-in-time - compiling (which speeds up matching), pcregrep automatically - makes use of this, unless it was explicitly disabled at build - time. This option can be used to disable the use of JIT at - run time. It is provided for testing and working round prob- - lems. It should never be needed in normal use. - - -o, --only-matching - Show only the part of the line that matched a pattern instead - of the whole line. In this mode, no context is shown. That - is, the -A, -B, and -C options are ignored. If there is more - than one match in a line, each of them is shown separately. - If -o is combined with -v (invert the sense of the match to - find non-matching lines), no output is generated, but the - return code is set appropriately. If the matched portion of - the line is empty, nothing is output unless the file name or - line number are being printed, in which case they are shown - on an otherwise empty line. This option is mutually exclusive - with --file-offsets and --line-offsets. - - -onumber, --only-matching=number - Show only the part of the line that matched the capturing - parentheses of the given number. Up to 32 capturing parenthe- - ses are supported, and -o0 is equivalent to -o without a num- - ber. Because these options can be given without an argument - (see above), if an argument is present, it must be given in - the same shell item, for example, -o3 or --only-matching=2. - The comments given for the non-argument case above also apply - to this case. If the specified capturing parentheses do not - exist in the pattern, or were not set in the match, nothing - is output unless the file name or line number are being - printed. - - If this option is given multiple times, multiple substrings - are output, in the order the options are given. For example, - -o3 -o1 -o3 causes the substrings matched by capturing paren- - theses 3 and 1 and then 3 again to be output. By default, - there is no separator (but see the next option). - - --om-separator=text - Specify a separating string for multiple occurrences of -o. - The default is an empty string. Separating strings are never - coloured. - - -q, --quiet - Work quietly, that is, display nothing except error messages. - The exit status indicates whether or not any matches were - found. - - -r, --recursive - If any given path is a directory, recursively scan the files - it contains, taking note of any --include and --exclude set- - tings. By default, a directory is read as a normal file; in - some operating systems this gives an immediate end-of-file. - This option is a shorthand for setting the -d option to - "recurse". - - --recursion-limit=number - See --match-limit above. - - -s, --no-messages - Suppress error messages about non-existent or unreadable - files. Such files are quietly skipped. However, the return - code is still 2, even if matches were found in other files. - - -u, --utf-8 - Operate in UTF-8 mode. This option is available only if PCRE - has been compiled with UTF-8 support. All patterns (including - those for any --exclude and --include options) and all sub- - ject lines that are scanned must be valid strings of UTF-8 - characters. - - -V, --version - Write the version numbers of pcregrep and the PCRE library to - the standard output and then exit. Anything else on the com- - mand line is ignored. - - -v, --invert-match - Invert the sense of the match, so that lines which do not - match any of the patterns are the ones that are found. - - -w, --word-regex, --word-regexp - Force the patterns to match only whole words. This is equiva- - lent to having \b at the start and end of the pattern. This - option applies only to the patterns that are matched against - the contents of files; it does not apply to patterns speci- - fied by any of the --include or --exclude options. - - -x, --line-regex, --line-regexp - Force the patterns to be anchored (each must start matching - at the beginning of a line) and in addition, require them to - match entire lines. This is equivalent to having ^ and $ - characters at the start and end of each alternative branch in - every pattern. This option applies only to the patterns that - are matched against the contents of files; it does not apply - to patterns specified by any of the --include or --exclude - options. - - -ENVIRONMENT VARIABLES - - The environment variables LC_ALL and LC_CTYPE are examined, in that - order, for a locale. The first one that is set is used. This can be - overridden by the --locale option. If no locale is set, the PCRE - library's default (usually the "C" locale) is used. - - -NEWLINES - - The -N (--newline) option allows pcregrep to scan files with different - newline conventions from the default. Any parts of the input files that - are written to the standard output are copied identically, with what- - ever newline sequences they have in the input. However, the setting of - this option does not affect the interpretation of files specified by - the -f, --exclude-from, or --include-from options, which are assumed to - use the operating system's standard newline sequence, nor does it - affect the way in which pcregrep writes informational messages to the - standard error and output streams. For these it uses the string "\n" to - indicate newlines, relying on the C I/O library to convert this to an - appropriate sequence. - - -OPTIONS COMPATIBILITY - - Many of the short and long forms of pcregrep's options are the same as - in the GNU grep program. Any long option of the form --xxx-regexp (GNU - terminology) is also available as --xxx-regex (PCRE terminology). How- - ever, the --file-list, --file-offsets, --include-dir, --line-offsets, - --locale, --match-limit, -M, --multiline, -N, --newline, --om-separa- - tor, --recursion-limit, -u, and --utf-8 options are specific to pcre- - grep, as is the use of the --only-matching option with a capturing - parentheses number. - - Although most of the common options work the same way, a few are dif- - ferent in pcregrep. For example, the --include option's argument is a - glob for GNU grep, but a regular expression for pcregrep. If both the - -c and -l options are given, GNU grep lists only file names, without - counts, but pcregrep gives the counts. - - -OPTIONS WITH DATA - - There are four different ways in which an option with data can be spec- - ified. If a short form option is used, the data may follow immedi- - ately, or (with one exception) in the next command line item. For exam- - ple: - - -f/some/file - -f /some/file - - The exception is the -o option, which may appear with or without data. - Because of this, if data is present, it must follow immediately in the - same item, for example -o3. - - If a long form option is used, the data may appear in the same command - line item, separated by an equals character, or (with two exceptions) - it may appear in the next command line item. For example: - - --file=/some/file - --file /some/file - - Note, however, that if you want to supply a file name beginning with ~ - as data in a shell command, and have the shell expand ~ to a home - directory, you must separate the file name from the option, because the - shell does not treat ~ specially unless it is at the start of an item. - - The exceptions to the above are the --colour (or --color) and --only- - matching options, for which the data is optional. If one of these - options does have data, it must be given in the first form, using an - equals character. Otherwise pcregrep will assume that it has no data. - - -MATCHING ERRORS - - It is possible to supply a regular expression that takes a very long - time to fail to match certain lines. Such patterns normally involve - nested indefinite repeats, for example: (a+)*\d when matched against a - line of a's with no final digit. The PCRE matching function has a - resource limit that causes it to abort in these circumstances. If this - happens, pcregrep outputs an error message and the line that caused the - problem to the standard error stream. If there are more than 20 such - errors, pcregrep gives up. - - The --match-limit option of pcregrep can be used to set the overall - resource limit; there is a second option called --recursion-limit that - sets a limit on the amount of memory (usually stack) that is used (see - the discussion of these options above). - - -DIAGNOSTICS - - Exit status is 0 if any matches were found, 1 if no matches were found, - and 2 for syntax errors, overlong lines, non-existent or inaccessible - files (even if matches were found in other files) or too many matching - errors. Using the -s option to suppress error messages about inaccessi- - ble files does not affect the return code. - - -SEE ALSO - - pcrepattern(3), pcresyntax(3), pcretest(1). - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 03 April 2014 - Copyright (c) 1997-2014 University of Cambridge. diff --git a/plugins/Pcre16/docs/doc/pcrejit.3 b/plugins/Pcre16/docs/doc/pcrejit.3 deleted file mode 100644 index 341403f7c8..0000000000 --- a/plugins/Pcre16/docs/doc/pcrejit.3 +++ /dev/null @@ -1,431 +0,0 @@ -.TH PCREJIT 3 "17 March 2013" "PCRE 8.33" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH "PCRE JUST-IN-TIME COMPILER SUPPORT" -.rs -.sp -Just-in-time compiling is a heavyweight optimization that can greatly speed up -pattern matching. However, it comes at the cost of extra processing before the -match is performed. Therefore, it is of most benefit when the same pattern is -going to be matched many times. This does not necessarily mean many calls of a -matching function; if the pattern is not anchored, matching attempts may take -place many times at various positions in the subject, even for a single call. -Therefore, if the subject string is very long, it may still pay to use JIT for -one-off matches. -.P -JIT support applies only to the traditional Perl-compatible matching function. -It does not apply when the DFA matching function is being used. The code for -this support was written by Zoltan Herczeg. -. -. -.SH "8-BIT, 16-BIT AND 32-BIT SUPPORT" -.rs -.sp -JIT support is available for all of the 8-bit, 16-bit and 32-bit PCRE -libraries. To keep this documentation simple, only the 8-bit interface is -described in what follows. If you are using the 16-bit library, substitute the -16-bit functions and 16-bit structures (for example, \fIpcre16_jit_stack\fP -instead of \fIpcre_jit_stack\fP). If you are using the 32-bit library, -substitute the 32-bit functions and 32-bit structures (for example, -\fIpcre32_jit_stack\fP instead of \fIpcre_jit_stack\fP). -. -. -.SH "AVAILABILITY OF JIT SUPPORT" -.rs -.sp -JIT support is an optional feature of PCRE. The "configure" option --enable-jit -(or equivalent CMake option) must be set when PCRE is built if you want to use -JIT. The support is limited to the following hardware platforms: -.sp - ARM v5, v7, and Thumb2 - Intel x86 32-bit and 64-bit - MIPS 32-bit - Power PC 32-bit and 64-bit - SPARC 32-bit (experimental) -.sp -If --enable-jit is set on an unsupported platform, compilation fails. -.P -A program that is linked with PCRE 8.20 or later can tell if JIT support is -available by calling \fBpcre_config()\fP with the PCRE_CONFIG_JIT option. The -result is 1 when JIT is available, and 0 otherwise. However, a simple program -does not need to check this in order to use JIT. The normal API is implemented -in a way that falls back to the interpretive code if JIT is not available. For -programs that need the best possible performance, there is also a "fast path" -API that is JIT-specific. -.P -If your program may sometimes be linked with versions of PCRE that are older -than 8.20, but you want to use JIT when it is available, you can test -the values of PCRE_MAJOR and PCRE_MINOR, or the existence of a JIT macro such -as PCRE_CONFIG_JIT, for compile-time control of your code. -. -. -.SH "SIMPLE USE OF JIT" -.rs -.sp -You have to do two things to make use of the JIT support in the simplest way: -.sp - (1) Call \fBpcre_study()\fP with the PCRE_STUDY_JIT_COMPILE option for - each compiled pattern, and pass the resulting \fBpcre_extra\fP block to - \fBpcre_exec()\fP. -.sp - (2) Use \fBpcre_free_study()\fP to free the \fBpcre_extra\fP block when it is - no longer needed, instead of just freeing it yourself. This ensures that - any JIT data is also freed. -.sp -For a program that may be linked with pre-8.20 versions of PCRE, you can insert -.sp - #ifndef PCRE_STUDY_JIT_COMPILE - #define PCRE_STUDY_JIT_COMPILE 0 - #endif -.sp -so that no option is passed to \fBpcre_study()\fP, and then use something like -this to free the study data: -.sp - #ifdef PCRE_CONFIG_JIT - pcre_free_study(study_ptr); - #else - pcre_free(study_ptr); - #endif -.sp -PCRE_STUDY_JIT_COMPILE requests the JIT compiler to generate code for complete -matches. If you want to run partial matches using the PCRE_PARTIAL_HARD or -PCRE_PARTIAL_SOFT options of \fBpcre_exec()\fP, you should set one or both of -the following options in addition to, or instead of, PCRE_STUDY_JIT_COMPILE -when you call \fBpcre_study()\fP: -.sp - PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE - PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE -.sp -The JIT compiler generates different optimized code for each of the three -modes (normal, soft partial, hard partial). When \fBpcre_exec()\fP is called, -the appropriate code is run if it is available. Otherwise, the pattern is -matched using interpretive code. -.P -In some circumstances you may need to call additional functions. These are -described in the section entitled -.\" HTML <a href="#stackcontrol"> -.\" </a> -"Controlling the JIT stack" -.\" -below. -.P -If JIT support is not available, PCRE_STUDY_JIT_COMPILE etc. are ignored, and -no JIT data is created. Otherwise, the compiled pattern is passed to the JIT -compiler, which turns it into machine code that executes much faster than the -normal interpretive code. When \fBpcre_exec()\fP is passed a \fBpcre_extra\fP -block containing a pointer to JIT code of the appropriate mode (normal or -hard/soft partial), it obeys that code instead of running the interpreter. The -result is identical, but the compiled JIT code runs much faster. -.P -There are some \fBpcre_exec()\fP options that are not supported for JIT -execution. There are also some pattern items that JIT cannot handle. Details -are given below. In both cases, execution automatically falls back to the -interpretive code. If you want to know whether JIT was actually used for a -particular match, you should arrange for a JIT callback function to be set up -as described in the section entitled -.\" HTML <a href="#stackcontrol"> -.\" </a> -"Controlling the JIT stack" -.\" -below, even if you do not need to supply a non-default JIT stack. Such a -callback function is called whenever JIT code is about to be obeyed. If the -execution options are not right for JIT execution, the callback function is not -obeyed. -.P -If the JIT compiler finds an unsupported item, no JIT data is generated. You -can find out if JIT execution is available after studying a pattern by calling -\fBpcre_fullinfo()\fP with the PCRE_INFO_JIT option. A result of 1 means that -JIT compilation was successful. A result of 0 means that JIT support is not -available, or the pattern was not studied with PCRE_STUDY_JIT_COMPILE etc., or -the JIT compiler was not able to handle the pattern. -.P -Once a pattern has been studied, with or without JIT, it can be used as many -times as you like for matching different subject strings. -. -. -.SH "UNSUPPORTED OPTIONS AND PATTERN ITEMS" -.rs -.sp -The only \fBpcre_exec()\fP options that are supported for JIT execution are -PCRE_NO_UTF8_CHECK, PCRE_NO_UTF16_CHECK, PCRE_NO_UTF32_CHECK, PCRE_NOTBOL, -PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, PCRE_PARTIAL_HARD, and -PCRE_PARTIAL_SOFT. -.P -The only unsupported pattern items are \eC (match a single data unit) when -running in a UTF mode, and a callout immediately before an assertion condition -in a conditional group. -. -. -.SH "RETURN VALUES FROM JIT EXECUTION" -.rs -.sp -When a pattern is matched using JIT execution, the return values are the same -as those given by the interpretive \fBpcre_exec()\fP code, with the addition of -one new error code: PCRE_ERROR_JIT_STACKLIMIT. This means that the memory used -for the JIT stack was insufficient. See -.\" HTML <a href="#stackcontrol"> -.\" </a> -"Controlling the JIT stack" -.\" -below for a discussion of JIT stack usage. For compatibility with the -interpretive \fBpcre_exec()\fP code, no more than two-thirds of the -\fIovector\fP argument is used for passing back captured substrings. -.P -The error code PCRE_ERROR_MATCHLIMIT is returned by the JIT code if searching a -very large pattern tree goes on for too long, as it is in the same circumstance -when JIT is not used, but the details of exactly what is counted are not the -same. The PCRE_ERROR_RECURSIONLIMIT error code is never returned by JIT -execution. -. -. -.SH "SAVING AND RESTORING COMPILED PATTERNS" -.rs -.sp -The code that is generated by the JIT compiler is architecture-specific, and is -also position dependent. For those reasons it cannot be saved (in a file or -database) and restored later like the bytecode and other data of a compiled -pattern. Saving and restoring compiled patterns is not something many people -do. More detail about this facility is given in the -.\" HREF -\fBpcreprecompile\fP -.\" -documentation. It should be possible to run \fBpcre_study()\fP on a saved and -restored pattern, and thereby recreate the JIT data, but because JIT -compilation uses significant resources, it is probably not worth doing this; -you might as well recompile the original pattern. -. -. -.\" HTML <a name="stackcontrol"></a> -.SH "CONTROLLING THE JIT STACK" -.rs -.sp -When the compiled JIT code runs, it needs a block of memory to use as a stack. -By default, it uses 32K on the machine stack. However, some large or -complicated patterns need more than this. The error PCRE_ERROR_JIT_STACKLIMIT -is given when there is not enough stack. Three functions are provided for -managing blocks of memory for use as JIT stacks. There is further discussion -about the use of JIT stacks in the section entitled -.\" HTML <a href="#stackcontrol"> -.\" </a> -"JIT stack FAQ" -.\" -below. -.P -The \fBpcre_jit_stack_alloc()\fP function creates a JIT stack. Its arguments -are a starting size and a maximum size, and it returns a pointer to an opaque -structure of type \fBpcre_jit_stack\fP, or NULL if there is an error. The -\fBpcre_jit_stack_free()\fP function can be used to free a stack that is no -longer needed. (For the technically minded: the address space is allocated by -mmap or VirtualAlloc.) -.P -JIT uses far less memory for recursion than the interpretive code, -and a maximum stack size of 512K to 1M should be more than enough for any -pattern. -.P -The \fBpcre_assign_jit_stack()\fP function specifies which stack JIT code -should use. Its arguments are as follows: -.sp - pcre_extra *extra - pcre_jit_callback callback - void *data -.sp -The \fIextra\fP argument must be the result of studying a pattern with -PCRE_STUDY_JIT_COMPILE etc. There are three cases for the values of the other -two options: -.sp - (1) If \fIcallback\fP is NULL and \fIdata\fP is NULL, an internal 32K block - on the machine stack is used. -.sp - (2) If \fIcallback\fP is NULL and \fIdata\fP is not NULL, \fIdata\fP must be - a valid JIT stack, the result of calling \fBpcre_jit_stack_alloc()\fP. -.sp - (3) If \fIcallback\fP is not NULL, it must point to a function that is - called with \fIdata\fP as an argument at the start of matching, in - order to set up a JIT stack. If the return from the callback - function is NULL, the internal 32K stack is used; otherwise the - return value must be a valid JIT stack, the result of calling - \fBpcre_jit_stack_alloc()\fP. -.sp -A callback function is obeyed whenever JIT code is about to be run; it is not -obeyed when \fBpcre_exec()\fP is called with options that are incompatible for -JIT execution. A callback function can therefore be used to determine whether a -match operation was executed by JIT or by the interpreter. -.P -You may safely use the same JIT stack for more than one pattern (either by -assigning directly or by callback), as long as the patterns are all matched -sequentially in the same thread. In a multithread application, if you do not -specify a JIT stack, or if you assign or pass back NULL from a callback, that -is thread-safe, because each thread has its own machine stack. However, if you -assign or pass back a non-NULL JIT stack, this must be a different stack for -each thread so that the application is thread-safe. -.P -Strictly speaking, even more is allowed. You can assign the same non-NULL stack -to any number of patterns as long as they are not used for matching by multiple -threads at the same time. For example, you can assign the same stack to all -compiled patterns, and use a global mutex in the callback to wait until the -stack is available for use. However, this is an inefficient solution, and not -recommended. -.P -This is a suggestion for how a multithreaded program that needs to set up -non-default JIT stacks might operate: -.sp - During thread initalization - thread_local_var = pcre_jit_stack_alloc(...) -.sp - During thread exit - pcre_jit_stack_free(thread_local_var) -.sp - Use a one-line callback function - return thread_local_var -.sp -All the functions described in this section do nothing if JIT is not available, -and \fBpcre_assign_jit_stack()\fP does nothing unless the \fBextra\fP argument -is non-NULL and points to a \fBpcre_extra\fP block that is the result of a -successful study with PCRE_STUDY_JIT_COMPILE etc. -. -. -.\" HTML <a name="stackfaq"></a> -.SH "JIT STACK FAQ" -.rs -.sp -(1) Why do we need JIT stacks? -.sp -PCRE (and JIT) is a recursive, depth-first engine, so it needs a stack where -the local data of the current node is pushed before checking its child nodes. -Allocating real machine stack on some platforms is difficult. For example, the -stack chain needs to be updated every time if we extend the stack on PowerPC. -Although it is possible, its updating time overhead decreases performance. So -we do the recursion in memory. -.P -(2) Why don't we simply allocate blocks of memory with \fBmalloc()\fP? -.sp -Modern operating systems have a nice feature: they can reserve an address space -instead of allocating memory. We can safely allocate memory pages inside this -address space, so the stack could grow without moving memory data (this is -important because of pointers). Thus we can allocate 1M address space, and use -only a single memory page (usually 4K) if that is enough. However, we can still -grow up to 1M anytime if needed. -.P -(3) Who "owns" a JIT stack? -.sp -The owner of the stack is the user program, not the JIT studied pattern or -anything else. The user program must ensure that if a stack is used by -\fBpcre_exec()\fP, (that is, it is assigned to the pattern currently running), -that stack must not be used by any other threads (to avoid overwriting the same -memory area). The best practice for multithreaded programs is to allocate a -stack for each thread, and return this stack through the JIT callback function. -.P -(4) When should a JIT stack be freed? -.sp -You can free a JIT stack at any time, as long as it will not be used by -\fBpcre_exec()\fP again. When you assign the stack to a pattern, only a pointer -is set. There is no reference counting or any other magic. You can free the -patterns and stacks in any order, anytime. Just \fIdo not\fP call -\fBpcre_exec()\fP with a pattern pointing to an already freed stack, as that -will cause SEGFAULT. (Also, do not free a stack currently used by -\fBpcre_exec()\fP in another thread). You can also replace the stack for a -pattern at any time. You can even free the previous stack before assigning a -replacement. -.P -(5) Should I allocate/free a stack every time before/after calling -\fBpcre_exec()\fP? -.sp -No, because this is too costly in terms of resources. However, you could -implement some clever idea which release the stack if it is not used in let's -say two minutes. The JIT callback can help to achieve this without keeping a -list of the currently JIT studied patterns. -.P -(6) OK, the stack is for long term memory allocation. But what happens if a -pattern causes stack overflow with a stack of 1M? Is that 1M kept until the -stack is freed? -.sp -Especially on embedded sytems, it might be a good idea to release memory -sometimes without freeing the stack. There is no API for this at the moment. -Probably a function call which returns with the currently allocated memory for -any stack and another which allows releasing memory (shrinking the stack) would -be a good idea if someone needs this. -.P -(7) This is too much of a headache. Isn't there any better solution for JIT -stack handling? -.sp -No, thanks to Windows. If POSIX threads were used everywhere, we could throw -out this complicated API. -. -. -.SH "EXAMPLE CODE" -.rs -.sp -This is a single-threaded example that specifies a JIT stack without using a -callback. -.sp - int rc; - int ovector[30]; - pcre *re; - pcre_extra *extra; - pcre_jit_stack *jit_stack; -.sp - re = pcre_compile(pattern, 0, &error, &erroffset, NULL); - /* Check for errors */ - extra = pcre_study(re, PCRE_STUDY_JIT_COMPILE, &error); - jit_stack = pcre_jit_stack_alloc(32*1024, 512*1024); - /* Check for error (NULL) */ - pcre_assign_jit_stack(extra, NULL, jit_stack); - rc = pcre_exec(re, extra, subject, length, 0, 0, ovector, 30); - /* Check results */ - pcre_free(re); - pcre_free_study(extra); - pcre_jit_stack_free(jit_stack); -.sp -. -. -.SH "JIT FAST PATH API" -.rs -.sp -Because the API described above falls back to interpreted execution when JIT is -not available, it is convenient for programs that are written for general use -in many environments. However, calling JIT via \fBpcre_exec()\fP does have a -performance impact. Programs that are written for use where JIT is known to be -available, and which need the best possible performance, can instead use a -"fast path" API to call JIT execution directly instead of calling -\fBpcre_exec()\fP (obviously only for patterns that have been successfully -studied by JIT). -.P -The fast path function is called \fBpcre_jit_exec()\fP, and it takes exactly -the same arguments as \fBpcre_exec()\fP, plus one additional argument that -must point to a JIT stack. The JIT stack arrangements described above do not -apply. The return values are the same as for \fBpcre_exec()\fP. -.P -When you call \fBpcre_exec()\fP, as well as testing for invalid options, a -number of other sanity checks are performed on the arguments. For example, if -the subject pointer is NULL, or its length is negative, an immediate error is -given. Also, unless PCRE_NO_UTF[8|16|32] is set, a UTF subject string is tested -for validity. In the interests of speed, these checks do not happen on the JIT -fast path, and if invalid data is passed, the result is undefined. -.P -Bypassing the sanity checks and the \fBpcre_exec()\fP wrapping can give -speedups of more than 10%. -. -. -.SH "SEE ALSO" -.rs -.sp -\fBpcreapi\fP(3) -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel (FAQ by Zoltan Herczeg) -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 17 March 2013 -Copyright (c) 1997-2013 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcrelimits.3 b/plugins/Pcre16/docs/doc/pcrelimits.3 deleted file mode 100644 index 423d6a2768..0000000000 --- a/plugins/Pcre16/docs/doc/pcrelimits.3 +++ /dev/null @@ -1,71 +0,0 @@ -.TH PCRELIMITS 3 "05 November 2013" "PCRE 8.34" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH "SIZE AND OTHER LIMITATIONS" -.rs -.sp -There are some size limitations in PCRE but it is hoped that they will never in -practice be relevant. -.P -The maximum length of a compiled pattern is approximately 64K data units (bytes -for the 8-bit library, 16-bit units for the 16-bit library, and 32-bit units for -the 32-bit library) if PCRE is compiled with the default internal linkage size, -which is 2 bytes for the 8-bit and 16-bit libraries, and 4 bytes for the 32-bit -library. If you want to process regular expressions that are truly enormous, -you can compile PCRE with an internal linkage size of 3 or 4 (when building the -16-bit or 32-bit library, 3 is rounded up to 4). See the \fBREADME\fP file in -the source distribution and the -.\" HREF -\fBpcrebuild\fP -.\" -documentation for details. In these cases the limit is substantially larger. -However, the speed of execution is slower. -.P -All values in repeating quantifiers must be less than 65536. -.P -There is no limit to the number of parenthesized subpatterns, but there can be -no more than 65535 capturing subpatterns. There is, however, a limit to the -depth of nesting of parenthesized subpatterns of all kinds. This is imposed in -order to limit the amount of system stack used at compile time. The limit can -be specified when PCRE is built; the default is 250. -.P -There is a limit to the number of forward references to subsequent subpatterns -of around 200,000. Repeated forward references with fixed upper limits, for -example, (?2){0,100} when subpattern number 2 is to the right, are included in -the count. There is no limit to the number of backward references. -.P -The maximum length of name for a named subpattern is 32 characters, and the -maximum number of named subpatterns is 10000. -.P -The maximum length of a name in a (*MARK), (*PRUNE), (*SKIP), or (*THEN) verb -is 255 for the 8-bit library and 65535 for the 16-bit and 32-bit libraries. -.P -The maximum length of a subject string is the largest positive number that an -integer variable can hold. However, when using the traditional matching -function, PCRE uses recursion to handle subpatterns and indefinite repetition. -This means that the available stack space may limit the size of a subject -string that can be processed by certain patterns. For a discussion of stack -issues, see the -.\" HREF -\fBpcrestack\fP -.\" -documentation. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 05 November 2013 -Copyright (c) 1997-2013 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcrematching.3 b/plugins/Pcre16/docs/doc/pcrematching.3 deleted file mode 100644 index 268baf9b8c..0000000000 --- a/plugins/Pcre16/docs/doc/pcrematching.3 +++ /dev/null @@ -1,214 +0,0 @@ -.TH PCREMATCHING 3 "12 November 2013" "PCRE 8.34" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH "PCRE MATCHING ALGORITHMS" -.rs -.sp -This document describes the two different algorithms that are available in PCRE -for matching a compiled regular expression against a given subject string. The -"standard" algorithm is the one provided by the \fBpcre_exec()\fP, -\fBpcre16_exec()\fP and \fBpcre32_exec()\fP functions. These work in the same -as as Perl's matching function, and provide a Perl-compatible matching operation. -The just-in-time (JIT) optimization that is described in the -.\" HREF -\fBpcrejit\fP -.\" -documentation is compatible with these functions. -.P -An alternative algorithm is provided by the \fBpcre_dfa_exec()\fP, -\fBpcre16_dfa_exec()\fP and \fBpcre32_dfa_exec()\fP functions; they operate in -a different way, and are not Perl-compatible. This alternative has advantages -and disadvantages compared with the standard algorithm, and these are described -below. -.P -When there is only one possible way in which a given subject string can match a -pattern, the two algorithms give the same answer. A difference arises, however, -when there are multiple possibilities. For example, if the pattern -.sp - ^<.*> -.sp -is matched against the string -.sp - <something> <something else> <something further> -.sp -there are three possible answers. The standard algorithm finds only one of -them, whereas the alternative algorithm finds all three. -. -. -.SH "REGULAR EXPRESSIONS AS TREES" -.rs -.sp -The set of strings that are matched by a regular expression can be represented -as a tree structure. An unlimited repetition in the pattern makes the tree of -infinite size, but it is still a tree. Matching the pattern to a given subject -string (from a given starting point) can be thought of as a search of the tree. -There are two ways to search a tree: depth-first and breadth-first, and these -correspond to the two matching algorithms provided by PCRE. -. -. -.SH "THE STANDARD MATCHING ALGORITHM" -.rs -.sp -In the terminology of Jeffrey Friedl's book "Mastering Regular -Expressions", the standard algorithm is an "NFA algorithm". It conducts a -depth-first search of the pattern tree. That is, it proceeds along a single -path through the tree, checking that the subject matches what is required. When -there is a mismatch, the algorithm tries any alternatives at the current point, -and if they all fail, it backs up to the previous branch point in the tree, and -tries the next alternative branch at that level. This often involves backing up -(moving to the left) in the subject string as well. The order in which -repetition branches are tried is controlled by the greedy or ungreedy nature of -the quantifier. -.P -If a leaf node is reached, a matching string has been found, and at that point -the algorithm stops. Thus, if there is more than one possible match, this -algorithm returns the first one that it finds. Whether this is the shortest, -the longest, or some intermediate length depends on the way the greedy and -ungreedy repetition quantifiers are specified in the pattern. -.P -Because it ends up with a single path through the tree, it is relatively -straightforward for this algorithm to keep track of the substrings that are -matched by portions of the pattern in parentheses. This provides support for -capturing parentheses and back references. -. -. -.SH "THE ALTERNATIVE MATCHING ALGORITHM" -.rs -.sp -This algorithm conducts a breadth-first search of the tree. Starting from the -first matching point in the subject, it scans the subject string from left to -right, once, character by character, and as it does this, it remembers all the -paths through the tree that represent valid matches. In Friedl's terminology, -this is a kind of "DFA algorithm", though it is not implemented as a -traditional finite state machine (it keeps multiple states active -simultaneously). -.P -Although the general principle of this matching algorithm is that it scans the -subject string only once, without backtracking, there is one exception: when a -lookaround assertion is encountered, the characters following or preceding the -current point have to be independently inspected. -.P -The scan continues until either the end of the subject is reached, or there are -no more unterminated paths. At this point, terminated paths represent the -different matching possibilities (if there are none, the match has failed). -Thus, if there is more than one possible match, this algorithm finds all of -them, and in particular, it finds the longest. The matches are returned in -decreasing order of length. There is an option to stop the algorithm after the -first match (which is necessarily the shortest) is found. -.P -Note that all the matches that are found start at the same point in the -subject. If the pattern -.sp - cat(er(pillar)?)? -.sp -is matched against the string "the caterpillar catchment", the result will be -the three strings "caterpillar", "cater", and "cat" that start at the fifth -character of the subject. The algorithm does not automatically move on to find -matches that start at later positions. -.P -PCRE's "auto-possessification" optimization usually applies to character -repeats at the end of a pattern (as well as internally). For example, the -pattern "a\ed+" is compiled as if it were "a\ed++" because there is no point -even considering the possibility of backtracking into the repeated digits. For -DFA matching, this means that only one possible match is found. If you really -do want multiple matches in such cases, either use an ungreedy repeat -("a\ed+?") or set the PCRE_NO_AUTO_POSSESS option when compiling. -.P -There are a number of features of PCRE regular expressions that are not -supported by the alternative matching algorithm. They are as follows: -.P -1. Because the algorithm finds all possible matches, the greedy or ungreedy -nature of repetition quantifiers is not relevant. Greedy and ungreedy -quantifiers are treated in exactly the same way. However, possessive -quantifiers can make a difference when what follows could also match what is -quantified, for example in a pattern like this: -.sp - ^a++\ew! -.sp -This pattern matches "aaab!" but not "aaa!", which would be matched by a -non-possessive quantifier. Similarly, if an atomic group is present, it is -matched as if it were a standalone pattern at the current point, and the -longest match is then "locked in" for the rest of the overall pattern. -.P -2. When dealing with multiple paths through the tree simultaneously, it is not -straightforward to keep track of captured substrings for the different matching -possibilities, and PCRE's implementation of this algorithm does not attempt to -do this. This means that no captured substrings are available. -.P -3. Because no substrings are captured, back references within the pattern are -not supported, and cause errors if encountered. -.P -4. For the same reason, conditional expressions that use a backreference as the -condition or test for a specific group recursion are not supported. -.P -5. Because many paths through the tree may be active, the \eK escape sequence, -which resets the start of the match when encountered (but may be on some paths -and not on others), is not supported. It causes an error if encountered. -.P -6. Callouts are supported, but the value of the \fIcapture_top\fP field is -always 1, and the value of the \fIcapture_last\fP field is always -1. -.P -7. The \eC escape sequence, which (in the standard algorithm) always matches a -single data unit, even in UTF-8, UTF-16 or UTF-32 modes, is not supported in -these modes, because the alternative algorithm moves through the subject string -one character (not data unit) at a time, for all active paths through the tree. -.P -8. Except for (*FAIL), the backtracking control verbs such as (*PRUNE) are not -supported. (*FAIL) is supported, and behaves like a failing negative assertion. -. -. -.SH "ADVANTAGES OF THE ALTERNATIVE ALGORITHM" -.rs -.sp -Using the alternative matching algorithm provides the following advantages: -.P -1. All possible matches (at a single point in the subject) are automatically -found, and in particular, the longest match is found. To find more than one -match using the standard algorithm, you have to do kludgy things with -callouts. -.P -2. Because the alternative algorithm scans the subject string just once, and -never needs to backtrack (except for lookbehinds), it is possible to pass very -long subject strings to the matching function in several pieces, checking for -partial matching each time. Although it is possible to do multi-segment -matching using the standard algorithm by retaining partially matched -substrings, it is more complicated. The -.\" HREF -\fBpcrepartial\fP -.\" -documentation gives details of partial matching and discusses multi-segment -matching. -. -. -.SH "DISADVANTAGES OF THE ALTERNATIVE ALGORITHM" -.rs -.sp -The alternative algorithm suffers from a number of disadvantages: -.P -1. It is substantially slower than the standard algorithm. This is partly -because it has to search for all possible matches, but is also because it is -less susceptible to optimization. -.P -2. Capturing parentheses and back references are not supported. -.P -3. Although atomic groups are supported, their use does not provide the -performance advantage that it does for the standard algorithm. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 12 November 2013 -Copyright (c) 1997-2012 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcrepartial.3 b/plugins/Pcre16/docs/doc/pcrepartial.3 deleted file mode 100644 index 14d0124f1c..0000000000 --- a/plugins/Pcre16/docs/doc/pcrepartial.3 +++ /dev/null @@ -1,476 +0,0 @@ -.TH PCREPARTIAL 3 "02 July 2013" "PCRE 8.34" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH "PARTIAL MATCHING IN PCRE" -.rs -.sp -In normal use of PCRE, if the subject string that is passed to a matching -function matches as far as it goes, but is too short to match the entire -pattern, PCRE_ERROR_NOMATCH is returned. There are circumstances where it might -be helpful to distinguish this case from other cases in which there is no -match. -.P -Consider, for example, an application where a human is required to type in data -for a field with specific formatting requirements. An example might be a date -in the form \fIddmmmyy\fP, defined by this pattern: -.sp - ^\ed?\ed(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\ed\ed$ -.sp -If the application sees the user's keystrokes one by one, and can check that -what has been typed so far is potentially valid, it is able to raise an error -as soon as a mistake is made, by beeping and not reflecting the character that -has been typed, for example. This immediate feedback is likely to be a better -user interface than a check that is delayed until the entire string has been -entered. Partial matching can also be useful when the subject string is very -long and is not all available at once. -.P -PCRE supports partial matching by means of the PCRE_PARTIAL_SOFT and -PCRE_PARTIAL_HARD options, which can be set when calling any of the matching -functions. For backwards compatibility, PCRE_PARTIAL is a synonym for -PCRE_PARTIAL_SOFT. The essential difference between the two options is whether -or not a partial match is preferred to an alternative complete match, though -the details differ between the two types of matching function. If both options -are set, PCRE_PARTIAL_HARD takes precedence. -.P -If you want to use partial matching with just-in-time optimized code, you must -call \fBpcre_study()\fP, \fBpcre16_study()\fP or \fBpcre32_study()\fP with one -or both of these options: -.sp - PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE - PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE -.sp -PCRE_STUDY_JIT_COMPILE should also be set if you are going to run non-partial -matches on the same pattern. If the appropriate JIT study mode has not been set -for a match, the interpretive matching code is used. -.P -Setting a partial matching option disables two of PCRE's standard -optimizations. PCRE remembers the last literal data unit in a pattern, and -abandons matching immediately if it is not present in the subject string. This -optimization cannot be used for a subject string that might match only -partially. If the pattern was studied, PCRE knows the minimum length of a -matching string, and does not bother to run the matching function on shorter -strings. This optimization is also disabled for partial matching. -. -. -.SH "PARTIAL MATCHING USING pcre_exec() OR pcre[16|32]_exec()" -.rs -.sp -A partial match occurs during a call to \fBpcre_exec()\fP or -\fBpcre[16|32]_exec()\fP when the end of the subject string is reached -successfully, but matching cannot continue because more characters are needed. -However, at least one character in the subject must have been inspected. This -character need not form part of the final matched string; lookbehind assertions -and the \eK escape sequence provide ways of inspecting characters before the -start of a matched substring. The requirement for inspecting at least one -character exists because an empty string can always be matched; without such a -restriction there would always be a partial match of an empty string at the end -of the subject. -.P -If there are at least two slots in the offsets vector when a partial match is -returned, the first slot is set to the offset of the earliest character that -was inspected. For convenience, the second offset points to the end of the -subject so that a substring can easily be identified. If there are at least -three slots in the offsets vector, the third slot is set to the offset of the -character where matching started. -.P -For the majority of patterns, the contents of the first and third slots will be -the same. However, for patterns that contain lookbehind assertions, or begin -with \eb or \eB, characters before the one where matching started may have been -inspected while carrying out the match. For example, consider this pattern: -.sp - /(?<=abc)123/ -.sp -This pattern matches "123", but only if it is preceded by "abc". If the subject -string is "xyzabc12", the first two offsets after a partial match are for the -substring "abc12", because all these characters were inspected. However, the -third offset is set to 6, because that is the offset where matching began. -.P -What happens when a partial match is identified depends on which of the two -partial matching options are set. -. -. -.SS "PCRE_PARTIAL_SOFT WITH pcre_exec() OR pcre[16|32]_exec()" -.rs -.sp -If PCRE_PARTIAL_SOFT is set when \fBpcre_exec()\fP or \fBpcre[16|32]_exec()\fP -identifies a partial match, the partial match is remembered, but matching -continues as normal, and other alternatives in the pattern are tried. If no -complete match can be found, PCRE_ERROR_PARTIAL is returned instead of -PCRE_ERROR_NOMATCH. -.P -This option is "soft" because it prefers a complete match over a partial match. -All the various matching items in a pattern behave as if the subject string is -potentially complete. For example, \ez, \eZ, and $ match at the end of the -subject, as normal, and for \eb and \eB the end of the subject is treated as a -non-alphanumeric. -.P -If there is more than one partial match, the first one that was found provides -the data that is returned. Consider this pattern: -.sp - /123\ew+X|dogY/ -.sp -If this is matched against the subject string "abc123dog", both -alternatives fail to match, but the end of the subject is reached during -matching, so PCRE_ERROR_PARTIAL is returned. The offsets are set to 3 and 9, -identifying "123dog" as the first partial match that was found. (In this -example, there are two partial matches, because "dog" on its own partially -matches the second alternative.) -. -. -.SS "PCRE_PARTIAL_HARD WITH pcre_exec() OR pcre[16|32]_exec()" -.rs -.sp -If PCRE_PARTIAL_HARD is set for \fBpcre_exec()\fP or \fBpcre[16|32]_exec()\fP, -PCRE_ERROR_PARTIAL is returned as soon as a partial match is found, without -continuing to search for possible complete matches. This option is "hard" -because it prefers an earlier partial match over a later complete match. For -this reason, the assumption is made that the end of the supplied subject string -may not be the true end of the available data, and so, if \ez, \eZ, \eb, \eB, -or $ are encountered at the end of the subject, the result is -PCRE_ERROR_PARTIAL, provided that at least one character in the subject has -been inspected. -.P -Setting PCRE_PARTIAL_HARD also affects the way UTF-8 and UTF-16 -subject strings are checked for validity. Normally, an invalid sequence -causes the error PCRE_ERROR_BADUTF8 or PCRE_ERROR_BADUTF16. However, in the -special case of a truncated character at the end of the subject, -PCRE_ERROR_SHORTUTF8 or PCRE_ERROR_SHORTUTF16 is returned when -PCRE_PARTIAL_HARD is set. -. -. -.SS "Comparing hard and soft partial matching" -.rs -.sp -The difference between the two partial matching options can be illustrated by a -pattern such as: -.sp - /dog(sbody)?/ -.sp -This matches either "dog" or "dogsbody", greedily (that is, it prefers the -longer string if possible). If it is matched against the string "dog" with -PCRE_PARTIAL_SOFT, it yields a complete match for "dog". However, if -PCRE_PARTIAL_HARD is set, the result is PCRE_ERROR_PARTIAL. On the other hand, -if the pattern is made ungreedy the result is different: -.sp - /dog(sbody)??/ -.sp -In this case the result is always a complete match because that is found first, -and matching never continues after finding a complete match. It might be easier -to follow this explanation by thinking of the two patterns like this: -.sp - /dog(sbody)?/ is the same as /dogsbody|dog/ - /dog(sbody)??/ is the same as /dog|dogsbody/ -.sp -The second pattern will never match "dogsbody", because it will always find the -shorter match first. -. -. -.SH "PARTIAL MATCHING USING pcre_dfa_exec() OR pcre[16|32]_dfa_exec()" -.rs -.sp -The DFA functions move along the subject string character by character, without -backtracking, searching for all possible matches simultaneously. If the end of -the subject is reached before the end of the pattern, there is the possibility -of a partial match, again provided that at least one character has been -inspected. -.P -When PCRE_PARTIAL_SOFT is set, PCRE_ERROR_PARTIAL is returned only if there -have been no complete matches. Otherwise, the complete matches are returned. -However, if PCRE_PARTIAL_HARD is set, a partial match takes precedence over any -complete matches. The portion of the string that was inspected when the longest -partial match was found is set as the first matching string, provided there are -at least two slots in the offsets vector. -.P -Because the DFA functions always search for all possible matches, and there is -no difference between greedy and ungreedy repetition, their behaviour is -different from the standard functions when PCRE_PARTIAL_HARD is set. Consider -the string "dog" matched against the ungreedy pattern shown above: -.sp - /dog(sbody)??/ -.sp -Whereas the standard functions stop as soon as they find the complete match for -"dog", the DFA functions also find the partial match for "dogsbody", and so -return that when PCRE_PARTIAL_HARD is set. -. -. -.SH "PARTIAL MATCHING AND WORD BOUNDARIES" -.rs -.sp -If a pattern ends with one of sequences \eb or \eB, which test for word -boundaries, partial matching with PCRE_PARTIAL_SOFT can give counter-intuitive -results. Consider this pattern: -.sp - /\ebcat\eb/ -.sp -This matches "cat", provided there is a word boundary at either end. If the -subject string is "the cat", the comparison of the final "t" with a following -character cannot take place, so a partial match is found. However, normal -matching carries on, and \eb matches at the end of the subject when the last -character is a letter, so a complete match is found. The result, therefore, is -\fInot\fP PCRE_ERROR_PARTIAL. Using PCRE_PARTIAL_HARD in this case does yield -PCRE_ERROR_PARTIAL, because then the partial match takes precedence. -. -. -.SH "FORMERLY RESTRICTED PATTERNS" -.rs -.sp -For releases of PCRE prior to 8.00, because of the way certain internal -optimizations were implemented in the \fBpcre_exec()\fP function, the -PCRE_PARTIAL option (predecessor of PCRE_PARTIAL_SOFT) could not be used with -all patterns. From release 8.00 onwards, the restrictions no longer apply, and -partial matching with can be requested for any pattern. -.P -Items that were formerly restricted were repeated single characters and -repeated metasequences. If PCRE_PARTIAL was set for a pattern that did not -conform to the restrictions, \fBpcre_exec()\fP returned the error code -PCRE_ERROR_BADPARTIAL (-13). This error code is no longer in use. The -PCRE_INFO_OKPARTIAL call to \fBpcre_fullinfo()\fP to find out if a compiled -pattern can be used for partial matching now always returns 1. -. -. -.SH "EXAMPLE OF PARTIAL MATCHING USING PCRETEST" -.rs -.sp -If the escape sequence \eP is present in a \fBpcretest\fP data line, the -PCRE_PARTIAL_SOFT option is used for the match. Here is a run of \fBpcretest\fP -that uses the date example quoted above: -.sp - re> /^\ed?\ed(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\ed\ed$/ - data> 25jun04\eP - 0: 25jun04 - 1: jun - data> 25dec3\eP - Partial match: 23dec3 - data> 3ju\eP - Partial match: 3ju - data> 3juj\eP - No match - data> j\eP - No match -.sp -The first data string is matched completely, so \fBpcretest\fP shows the -matched substrings. The remaining four strings do not match the complete -pattern, but the first two are partial matches. Similar output is obtained -if DFA matching is used. -.P -If the escape sequence \eP is present more than once in a \fBpcretest\fP data -line, the PCRE_PARTIAL_HARD option is set for the match. -. -. -.SH "MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre[16|32]_dfa_exec()" -.rs -.sp -When a partial match has been found using a DFA matching function, it is -possible to continue the match by providing additional subject data and calling -the function again with the same compiled regular expression, this time setting -the PCRE_DFA_RESTART option. You must pass the same working space as before, -because this is where details of the previous partial match are stored. Here is -an example using \fBpcretest\fP, using the \eR escape sequence to set the -PCRE_DFA_RESTART option (\eD specifies the use of the DFA matching function): -.sp - re> /^\ed?\ed(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\ed\ed$/ - data> 23ja\eP\eD - Partial match: 23ja - data> n05\eR\eD - 0: n05 -.sp -The first call has "23ja" as the subject, and requests partial matching; the -second call has "n05" as the subject for the continued (restarted) match. -Notice that when the match is complete, only the last part is shown; PCRE does -not retain the previously partially-matched string. It is up to the calling -program to do that if it needs to. -.P -That means that, for an unanchored pattern, if a continued match fails, it is -not possible to try again at a new starting point. All this facility is capable -of doing is continuing with the previous match attempt. In the previous -example, if the second set of data is "ug23" the result is no match, even -though there would be a match for "aug23" if the entire string were given at -once. Depending on the application, this may or may not be what you want. -The only way to allow for starting again at the next character is to retain the -matched part of the subject and try a new complete match. -.P -You can set the PCRE_PARTIAL_SOFT or PCRE_PARTIAL_HARD options with -PCRE_DFA_RESTART to continue partial matching over multiple segments. This -facility can be used to pass very long subject strings to the DFA matching -functions. -. -. -.SH "MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre[16|32]_exec()" -.rs -.sp -From release 8.00, the standard matching functions can also be used to do -multi-segment matching. Unlike the DFA functions, it is not possible to -restart the previous match with a new segment of data. Instead, new data must -be added to the previous subject string, and the entire match re-run, starting -from the point where the partial match occurred. Earlier data can be discarded. -.P -It is best to use PCRE_PARTIAL_HARD in this situation, because it does not -treat the end of a segment as the end of the subject when matching \ez, \eZ, -\eb, \eB, and $. Consider an unanchored pattern that matches dates: -.sp - re> /\ed?\ed(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\ed\ed/ - data> The date is 23ja\eP\eP - Partial match: 23ja -.sp -At this stage, an application could discard the text preceding "23ja", add on -text from the next segment, and call the matching function again. Unlike the -DFA matching functions, the entire matching string must always be available, -and the complete matching process occurs for each call, so more memory and more -processing time is needed. -.P -\fBNote:\fP If the pattern contains lookbehind assertions, or \eK, or starts -with \eb or \eB, the string that is returned for a partial match includes -characters that precede the start of what would be returned for a complete -match, because it contains all the characters that were inspected during the -partial match. -. -. -.SH "ISSUES WITH MULTI-SEGMENT MATCHING" -.rs -.sp -Certain types of pattern may give problems with multi-segment matching, -whichever matching function is used. -.P -1. If the pattern contains a test for the beginning of a line, you need to pass -the PCRE_NOTBOL option when the subject string for any call does start at the -beginning of a line. There is also a PCRE_NOTEOL option, but in practice when -doing multi-segment matching you should be using PCRE_PARTIAL_HARD, which -includes the effect of PCRE_NOTEOL. -.P -2. Lookbehind assertions that have already been obeyed are catered for in the -offsets that are returned for a partial match. However a lookbehind assertion -later in the pattern could require even earlier characters to be inspected. You -can handle this case by using the PCRE_INFO_MAXLOOKBEHIND option of the -\fBpcre_fullinfo()\fP or \fBpcre[16|32]_fullinfo()\fP functions to obtain the -length of the longest lookbehind in the pattern. This length is given in -characters, not bytes. If you always retain at least that many characters -before the partially matched string, all should be well. (Of course, near the -start of the subject, fewer characters may be present; in that case all -characters should be retained.) -.P -From release 8.33, there is a more accurate way of deciding which characters to -retain. Instead of subtracting the length of the longest lookbehind from the -earliest inspected character (\fIoffsets[0]\fP), the match start position -(\fIoffsets[2]\fP) should be used, and the next match attempt started at the -\fIoffsets[2]\fP character by setting the \fIstartoffset\fP argument of -\fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP. -.P -For example, if the pattern "(?<=123)abc" is partially -matched against the string "xx123a", the three offset values returned are 2, 6, -and 5. This indicates that the matching process that gave a partial match -started at offset 5, but the characters "123a" were all inspected. The maximum -lookbehind for that pattern is 3, so taking that away from 5 shows that we need -only keep "123a", and the next match attempt can be started at offset 3 (that -is, at "a") when further characters have been added. When the match start is -not the earliest inspected character, \fBpcretest\fP shows it explicitly: -.sp - re> "(?<=123)abc" - data> xx123a\eP\eP - Partial match at offset 5: 123a -.P -3. Because a partial match must always contain at least one character, what -might be considered a partial match of an empty string actually gives a "no -match" result. For example: -.sp - re> /c(?<=abc)x/ - data> ab\eP - No match -.sp -If the next segment begins "cx", a match should be found, but this will only -happen if characters from the previous segment are retained. For this reason, a -"no match" result should be interpreted as "partial match of an empty string" -when the pattern contains lookbehinds. -.P -4. Matching a subject string that is split into multiple segments may not -always produce exactly the same result as matching over one single long string, -especially when PCRE_PARTIAL_SOFT is used. The section "Partial Matching and -Word Boundaries" above describes an issue that arises if the pattern ends with -\eb or \eB. Another kind of difference may occur when there are multiple -matching possibilities, because (for PCRE_PARTIAL_SOFT) a partial match result -is given only when there are no completed matches. This means that as soon as -the shortest match has been found, continuation to a new subject segment is no -longer possible. Consider again this \fBpcretest\fP example: -.sp - re> /dog(sbody)?/ - data> dogsb\eP - 0: dog - data> do\eP\eD - Partial match: do - data> gsb\eR\eP\eD - 0: g - data> dogsbody\eD - 0: dogsbody - 1: dog -.sp -The first data line passes the string "dogsb" to a standard matching function, -setting the PCRE_PARTIAL_SOFT option. Although the string is a partial match -for "dogsbody", the result is not PCRE_ERROR_PARTIAL, because the shorter -string "dog" is a complete match. Similarly, when the subject is presented to -a DFA matching function in several parts ("do" and "gsb" being the first two) -the match stops when "dog" has been found, and it is not possible to continue. -On the other hand, if "dogsbody" is presented as a single string, a DFA -matching function finds both matches. -.P -Because of these problems, it is best to use PCRE_PARTIAL_HARD when matching -multi-segment data. The example above then behaves differently: -.sp - re> /dog(sbody)?/ - data> dogsb\eP\eP - Partial match: dogsb - data> do\eP\eD - Partial match: do - data> gsb\eR\eP\eP\eD - Partial match: gsb -.sp -5. Patterns that contain alternatives at the top level which do not all start -with the same pattern item may not work as expected when PCRE_DFA_RESTART is -used. For example, consider this pattern: -.sp - 1234|3789 -.sp -If the first part of the subject is "ABC123", a partial match of the first -alternative is found at offset 3. There is no partial match for the second -alternative, because such a match does not start at the same point in the -subject string. Attempting to continue with the string "7890" does not yield a -match because only those alternatives that match at one point in the subject -are remembered. The problem arises because the start of the second alternative -matches within the first alternative. There is no problem with anchored -patterns or patterns such as: -.sp - 1234|ABCD -.sp -where no string can be a partial match for both alternatives. This is not a -problem if a standard matching function is used, because the entire match has -to be rerun each time: -.sp - re> /1234|3789/ - data> ABC123\eP\eP - Partial match: 123 - data> 1237890 - 0: 3789 -.sp -Of course, instead of using PCRE_DFA_RESTART, the same technique of re-running -the entire match can also be used with the DFA matching functions. Another -possibility is to work with two buffers. If a partial match at offset \fIn\fP -in the first buffer is followed by "no match" when PCRE_DFA_RESTART is used on -the second buffer, you can then try a new match starting at offset \fIn+1\fP in -the first buffer. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 02 July 2013 -Copyright (c) 1997-2013 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcrepattern.3 b/plugins/Pcre16/docs/doc/pcrepattern.3 deleted file mode 100644 index f1c45cda5d..0000000000 --- a/plugins/Pcre16/docs/doc/pcrepattern.3 +++ /dev/null @@ -1,3265 +0,0 @@ -.TH PCREPATTERN 3 "08 January 2014" "PCRE 8.35" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH "PCRE REGULAR EXPRESSION DETAILS" -.rs -.sp -The syntax and semantics of the regular expressions that are supported by PCRE -are described in detail below. There is a quick-reference syntax summary in the -.\" HREF -\fBpcresyntax\fP -.\" -page. PCRE tries to match Perl syntax and semantics as closely as it can. PCRE -also supports some alternative regular expression syntax (which does not -conflict with the Perl syntax) in order to provide some compatibility with -regular expressions in Python, .NET, and Oniguruma. -.P -Perl's regular expressions are described in its own documentation, and -regular expressions in general are covered in a number of books, some of which -have copious examples. Jeffrey Friedl's "Mastering Regular Expressions", -published by O'Reilly, covers regular expressions in great detail. This -description of PCRE's regular expressions is intended as reference material. -.P -This document discusses the patterns that are supported by PCRE when one its -main matching functions, \fBpcre_exec()\fP (8-bit) or \fBpcre[16|32]_exec()\fP -(16- or 32-bit), is used. PCRE also has alternative matching functions, -\fBpcre_dfa_exec()\fP and \fBpcre[16|32_dfa_exec()\fP, which match using a -different algorithm that is not Perl-compatible. Some of the features discussed -below are not available when DFA matching is used. The advantages and -disadvantages of the alternative functions, and how they differ from the normal -functions, are discussed in the -.\" HREF -\fBpcrematching\fP -.\" -page. -. -. -.SH "SPECIAL START-OF-PATTERN ITEMS" -.rs -.sp -A number of options that can be passed to \fBpcre_compile()\fP can also be set -by special items at the start of a pattern. These are not Perl-compatible, but -are provided to make these options accessible to pattern writers who are not -able to change the program that processes the pattern. Any number of these -items may appear, but they must all be together right at the start of the -pattern string, and the letters must be in upper case. -. -. -.SS "UTF support" -.rs -.sp -The original operation of PCRE was on strings of one-byte characters. However, -there is now also support for UTF-8 strings in the original library, an -extra library that supports 16-bit and UTF-16 character strings, and a -third library that supports 32-bit and UTF-32 character strings. To use these -features, PCRE must be built to include appropriate support. When using UTF -strings you must either call the compiling function with the PCRE_UTF8, -PCRE_UTF16, or PCRE_UTF32 option, or the pattern must start with one of -these special sequences: -.sp - (*UTF8) - (*UTF16) - (*UTF32) - (*UTF) -.sp -(*UTF) is a generic sequence that can be used with any of the libraries. -Starting a pattern with such a sequence is equivalent to setting the relevant -option. How setting a UTF mode affects pattern matching is mentioned in several -places below. There is also a summary of features in the -.\" HREF -\fBpcreunicode\fP -.\" -page. -.P -Some applications that allow their users to supply patterns may wish to -restrict them to non-UTF data for security reasons. If the PCRE_NEVER_UTF -option is set at compile time, (*UTF) etc. are not allowed, and their -appearance causes an error. -. -. -.SS "Unicode property support" -.rs -.sp -Another special sequence that may appear at the start of a pattern is (*UCP). -This has the same effect as setting the PCRE_UCP option: it causes sequences -such as \ed and \ew to use Unicode properties to determine character types, -instead of recognizing only characters with codes less than 128 via a lookup -table. -. -. -.SS "Disabling auto-possessification" -.rs -.sp -If a pattern starts with (*NO_AUTO_POSSESS), it has the same effect as setting -the PCRE_NO_AUTO_POSSESS option at compile time. This stops PCRE from making -quantifiers possessive when what follows cannot match the repeated item. For -example, by default a+b is treated as a++b. For more details, see the -.\" HREF -\fBpcreapi\fP -.\" -documentation. -. -. -.SS "Disabling start-up optimizations" -.rs -.sp -If a pattern starts with (*NO_START_OPT), it has the same effect as setting the -PCRE_NO_START_OPTIMIZE option either at compile or matching time. This disables -several optimizations for quickly reaching "no match" results. For more -details, see the -.\" HREF -\fBpcreapi\fP -.\" -documentation. -. -. -.\" HTML <a name="newlines"></a> -.SS "Newline conventions" -.rs -.sp -PCRE supports five different conventions for indicating line breaks in -strings: a single CR (carriage return) character, a single LF (linefeed) -character, the two-character sequence CRLF, any of the three preceding, or any -Unicode newline sequence. The -.\" HREF -\fBpcreapi\fP -.\" -page has -.\" HTML <a href="pcreapi.html#newlines"> -.\" </a> -further discussion -.\" -about newlines, and shows how to set the newline convention in the -\fIoptions\fP arguments for the compiling and matching functions. -.P -It is also possible to specify a newline convention by starting a pattern -string with one of the following five sequences: -.sp - (*CR) carriage return - (*LF) linefeed - (*CRLF) carriage return, followed by linefeed - (*ANYCRLF) any of the three above - (*ANY) all Unicode newline sequences -.sp -These override the default and the options given to the compiling function. For -example, on a Unix system where LF is the default newline sequence, the pattern -.sp - (*CR)a.b -.sp -changes the convention to CR. That pattern matches "a\enb" because LF is no -longer a newline. If more than one of these settings is present, the last one -is used. -.P -The newline convention affects where the circumflex and dollar assertions are -true. It also affects the interpretation of the dot metacharacter when -PCRE_DOTALL is not set, and the behaviour of \eN. However, it does not affect -what the \eR escape sequence matches. By default, this is any Unicode newline -sequence, for Perl compatibility. However, this can be changed; see the -description of \eR in the section entitled -.\" HTML <a href="#newlineseq"> -.\" </a> -"Newline sequences" -.\" -below. A change of \eR setting can be combined with a change of newline -convention. -. -. -.SS "Setting match and recursion limits" -.rs -.sp -The caller of \fBpcre_exec()\fP can set a limit on the number of times the -internal \fBmatch()\fP function is called and on the maximum depth of -recursive calls. These facilities are provided to catch runaway matches that -are provoked by patterns with huge matching trees (a typical example is a -pattern with nested unlimited repeats) and to avoid running out of system stack -by too much recursion. When one of these limits is reached, \fBpcre_exec()\fP -gives an error return. The limits can also be set by items at the start of the -pattern of the form -.sp - (*LIMIT_MATCH=d) - (*LIMIT_RECURSION=d) -.sp -where d is any number of decimal digits. However, the value of the setting must -be less than the value set (or defaulted) by the caller of \fBpcre_exec()\fP -for it to have any effect. In other words, the pattern writer can lower the -limits set by the programmer, but not raise them. If there is more than one -setting of one of these limits, the lower value is used. -. -. -.SH "EBCDIC CHARACTER CODES" -.rs -.sp -PCRE can be compiled to run in an environment that uses EBCDIC as its character -code rather than ASCII or Unicode (typically a mainframe system). In the -sections below, character code values are ASCII or Unicode; in an EBCDIC -environment these characters may have different code values, and there are no -code points greater than 255. -. -. -.SH "CHARACTERS AND METACHARACTERS" -.rs -.sp -A regular expression is a pattern that is matched against a subject string from -left to right. Most characters stand for themselves in a pattern, and match the -corresponding characters in the subject. As a trivial example, the pattern -.sp - The quick brown fox -.sp -matches a portion of a subject string that is identical to itself. When -caseless matching is specified (the PCRE_CASELESS option), letters are matched -independently of case. In a UTF mode, PCRE always understands the concept of -case for characters whose values are less than 128, so caseless matching is -always possible. For characters with higher values, the concept of case is -supported if PCRE is compiled with Unicode property support, but not otherwise. -If you want to use caseless matching for characters 128 and above, you must -ensure that PCRE is compiled with Unicode property support as well as with -UTF support. -.P -The power of regular expressions comes from the ability to include alternatives -and repetitions in the pattern. These are encoded in the pattern by the use of -\fImetacharacters\fP, which do not stand for themselves but instead are -interpreted in some special way. -.P -There are two different sets of metacharacters: those that are recognized -anywhere in the pattern except within square brackets, and those that are -recognized within square brackets. Outside square brackets, the metacharacters -are as follows: -.sp - \e general escape character with several uses - ^ assert start of string (or line, in multiline mode) - $ assert end of string (or line, in multiline mode) - . match any character except newline (by default) - [ start character class definition - | start of alternative branch - ( start subpattern - ) end subpattern - ? extends the meaning of ( - also 0 or 1 quantifier - also quantifier minimizer - * 0 or more quantifier - + 1 or more quantifier - also "possessive quantifier" - { start min/max quantifier -.sp -Part of a pattern that is in square brackets is called a "character class". In -a character class the only metacharacters are: -.sp - \e general escape character - ^ negate the class, but only if the first character - - indicates character range -.\" JOIN - [ POSIX character class (only if followed by POSIX - syntax) - ] terminates the character class -.sp -The following sections describe the use of each of the metacharacters. -. -. -.SH BACKSLASH -.rs -.sp -The backslash character has several uses. Firstly, if it is followed by a -character that is not a number or a letter, it takes away any special meaning -that character may have. This use of backslash as an escape character applies -both inside and outside character classes. -.P -For example, if you want to match a * character, you write \e* in the pattern. -This escaping action applies whether or not the following character would -otherwise be interpreted as a metacharacter, so it is always safe to precede a -non-alphanumeric with backslash to specify that it stands for itself. In -particular, if you want to match a backslash, you write \e\e. -.P -In a UTF mode, only ASCII numbers and letters have any special meaning after a -backslash. All other characters (in particular, those whose codepoints are -greater than 127) are treated as literals. -.P -If a pattern is compiled with the PCRE_EXTENDED option, most white space in the -pattern (other than in a character class), and characters between a # outside a -character class and the next newline, inclusive, are ignored. An escaping -backslash can be used to include a white space or # character as part of the -pattern. -.P -If you want to remove the special meaning from a sequence of characters, you -can do so by putting them between \eQ and \eE. This is different from Perl in -that $ and @ are handled as literals in \eQ...\eE sequences in PCRE, whereas in -Perl, $ and @ cause variable interpolation. Note the following examples: -.sp - Pattern PCRE matches Perl matches -.sp -.\" JOIN - \eQabc$xyz\eE abc$xyz abc followed by the - contents of $xyz - \eQabc\e$xyz\eE abc\e$xyz abc\e$xyz - \eQabc\eE\e$\eQxyz\eE abc$xyz abc$xyz -.sp -The \eQ...\eE sequence is recognized both inside and outside character classes. -An isolated \eE that is not preceded by \eQ is ignored. If \eQ is not followed -by \eE later in the pattern, the literal interpretation continues to the end of -the pattern (that is, \eE is assumed at the end). If the isolated \eQ is inside -a character class, this causes an error, because the character class is not -terminated. -. -. -.\" HTML <a name="digitsafterbackslash"></a> -.SS "Non-printing characters" -.rs -.sp -A second use of backslash provides a way of encoding non-printing characters -in patterns in a visible manner. There is no restriction on the appearance of -non-printing characters, apart from the binary zero that terminates a pattern, -but when a pattern is being prepared by text editing, it is often easier to use -one of the following escape sequences than the binary character it represents: -.sp - \ea alarm, that is, the BEL character (hex 07) - \ecx "control-x", where x is any ASCII character - \ee escape (hex 1B) - \ef form feed (hex 0C) - \en linefeed (hex 0A) - \er carriage return (hex 0D) - \et tab (hex 09) - \e0dd character with octal code 0dd - \eddd character with octal code ddd, or back reference - \eo{ddd..} character with octal code ddd.. - \exhh character with hex code hh - \ex{hhh..} character with hex code hhh.. (non-JavaScript mode) - \euhhhh character with hex code hhhh (JavaScript mode only) -.sp -The precise effect of \ecx on ASCII characters is as follows: if x is a lower -case letter, it is converted to upper case. Then bit 6 of the character (hex -40) is inverted. Thus \ecA to \ecZ become hex 01 to hex 1A (A is 41, Z is 5A), -but \ec{ becomes hex 3B ({ is 7B), and \ec; becomes hex 7B (; is 3B). If the -data item (byte or 16-bit value) following \ec has a value greater than 127, a -compile-time error occurs. This locks out non-ASCII characters in all modes. -.P -The \ec facility was designed for use with ASCII characters, but with the -extension to Unicode it is even less useful than it once was. It is, however, -recognized when PCRE is compiled in EBCDIC mode, where data items are always -bytes. In this mode, all values are valid after \ec. If the next character is a -lower case letter, it is converted to upper case. Then the 0xc0 bits of the -byte are inverted. Thus \ecA becomes hex 01, as in ASCII (A is C1), but because -the EBCDIC letters are disjoint, \ecZ becomes hex 29 (Z is E9), and other -characters also generate different values. -.P -After \e0 up to two further octal digits are read. If there are fewer than two -digits, just those that are present are used. Thus the sequence \e0\ex\e07 -specifies two binary zeros followed by a BEL character (code value 7). Make -sure you supply two digits after the initial zero if the pattern character that -follows is itself an octal digit. -.P -The escape \eo must be followed by a sequence of octal digits, enclosed in -braces. An error occurs if this is not the case. This escape is a recent -addition to Perl; it provides way of specifying character code points as octal -numbers greater than 0777, and it also allows octal numbers and back references -to be unambiguously specified. -.P -For greater clarity and unambiguity, it is best to avoid following \e by a -digit greater than zero. Instead, use \eo{} or \ex{} to specify character -numbers, and \eg{} to specify back references. The following paragraphs -describe the old, ambiguous syntax. -.P -The handling of a backslash followed by a digit other than 0 is complicated, -and Perl has changed in recent releases, causing PCRE also to change. Outside a -character class, PCRE reads the digit and any following digits as a decimal -number. If the number is less than 8, or if there have been at least that many -previous capturing left parentheses in the expression, the entire sequence is -taken as a \fIback reference\fP. A description of how this works is given -.\" HTML <a href="#backreferences"> -.\" </a> -later, -.\" -following the discussion of -.\" HTML <a href="#subpattern"> -.\" </a> -parenthesized subpatterns. -.\" -.P -Inside a character class, or if the decimal number following \e is greater than -7 and there have not been that many capturing subpatterns, PCRE handles \e8 and -\e9 as the literal characters "8" and "9", and otherwise re-reads up to three -octal digits following the backslash, using them to generate a data character. -Any subsequent digits stand for themselves. For example: -.sp - \e040 is another way of writing an ASCII space -.\" JOIN - \e40 is the same, provided there are fewer than 40 - previous capturing subpatterns - \e7 is always a back reference -.\" JOIN - \e11 might be a back reference, or another way of - writing a tab - \e011 is always a tab - \e0113 is a tab followed by the character "3" -.\" JOIN - \e113 might be a back reference, otherwise the - character with octal code 113 -.\" JOIN - \e377 might be a back reference, otherwise - the value 255 (decimal) -.\" JOIN - \e81 is either a back reference, or the two - characters "8" and "1" -.sp -Note that octal values of 100 or greater that are specified using this syntax -must not be introduced by a leading zero, because no more than three octal -digits are ever read. -.P -By default, after \ex that is not followed by {, from zero to two hexadecimal -digits are read (letters can be in upper or lower case). Any number of -hexadecimal digits may appear between \ex{ and }. If a character other than -a hexadecimal digit appears between \ex{ and }, or if there is no terminating -}, an error occurs. -.P -If the PCRE_JAVASCRIPT_COMPAT option is set, the interpretation of \ex is -as just described only when it is followed by two hexadecimal digits. -Otherwise, it matches a literal "x" character. In JavaScript mode, support for -code points greater than 256 is provided by \eu, which must be followed by -four hexadecimal digits; otherwise it matches a literal "u" character. -.P -Characters whose value is less than 256 can be defined by either of the two -syntaxes for \ex (or by \eu in JavaScript mode). There is no difference in the -way they are handled. For example, \exdc is exactly the same as \ex{dc} (or -\eu00dc in JavaScript mode). -. -. -.SS "Constraints on character values" -.rs -.sp -Characters that are specified using octal or hexadecimal numbers are -limited to certain values, as follows: -.sp - 8-bit non-UTF mode less than 0x100 - 8-bit UTF-8 mode less than 0x10ffff and a valid codepoint - 16-bit non-UTF mode less than 0x10000 - 16-bit UTF-16 mode less than 0x10ffff and a valid codepoint - 32-bit non-UTF mode less than 0x100000000 - 32-bit UTF-32 mode less than 0x10ffff and a valid codepoint -.sp -Invalid Unicode codepoints are the range 0xd800 to 0xdfff (the so-called -"surrogate" codepoints), and 0xffef. -. -. -.SS "Escape sequences in character classes" -.rs -.sp -All the sequences that define a single character value can be used both inside -and outside character classes. In addition, inside a character class, \eb is -interpreted as the backspace character (hex 08). -.P -\eN is not allowed in a character class. \eB, \eR, and \eX are not special -inside a character class. Like other unrecognized escape sequences, they are -treated as the literal characters "B", "R", and "X" by default, but cause an -error if the PCRE_EXTRA option is set. Outside a character class, these -sequences have different meanings. -. -. -.SS "Unsupported escape sequences" -.rs -.sp -In Perl, the sequences \el, \eL, \eu, and \eU are recognized by its string -handler and used to modify the case of following characters. By default, PCRE -does not support these escape sequences. However, if the PCRE_JAVASCRIPT_COMPAT -option is set, \eU matches a "U" character, and \eu can be used to define a -character by code point, as described in the previous section. -. -. -.SS "Absolute and relative back references" -.rs -.sp -The sequence \eg followed by an unsigned or a negative number, optionally -enclosed in braces, is an absolute or relative back reference. A named back -reference can be coded as \eg{name}. Back references are discussed -.\" HTML <a href="#backreferences"> -.\" </a> -later, -.\" -following the discussion of -.\" HTML <a href="#subpattern"> -.\" </a> -parenthesized subpatterns. -.\" -. -. -.SS "Absolute and relative subroutine calls" -.rs -.sp -For compatibility with Oniguruma, the non-Perl syntax \eg followed by a name or -a number enclosed either in angle brackets or single quotes, is an alternative -syntax for referencing a subpattern as a "subroutine". Details are discussed -.\" HTML <a href="#onigurumasubroutines"> -.\" </a> -later. -.\" -Note that \eg{...} (Perl syntax) and \eg<...> (Oniguruma syntax) are \fInot\fP -synonymous. The former is a back reference; the latter is a -.\" HTML <a href="#subpatternsassubroutines"> -.\" </a> -subroutine -.\" -call. -. -. -.\" HTML <a name="genericchartypes"></a> -.SS "Generic character types" -.rs -.sp -Another use of backslash is for specifying generic character types: -.sp - \ed any decimal digit - \eD any character that is not a decimal digit - \eh any horizontal white space character - \eH any character that is not a horizontal white space character - \es any white space character - \eS any character that is not a white space character - \ev any vertical white space character - \eV any character that is not a vertical white space character - \ew any "word" character - \eW any "non-word" character -.sp -There is also the single sequence \eN, which matches a non-newline character. -This is the same as -.\" HTML <a href="#fullstopdot"> -.\" </a> -the "." metacharacter -.\" -when PCRE_DOTALL is not set. Perl also uses \eN to match characters by name; -PCRE does not support this. -.P -Each pair of lower and upper case escape sequences partitions the complete set -of characters into two disjoint sets. Any given character matches one, and only -one, of each pair. The sequences can appear both inside and outside character -classes. They each match one character of the appropriate type. If the current -matching point is at the end of the subject string, all of them fail, because -there is no character to match. -.P -For compatibility with Perl, \es did not used to match the VT character (code -11), which made it different from the the POSIX "space" class. However, Perl -added VT at release 5.18, and PCRE followed suit at release 8.34. The default -\es characters are now HT (9), LF (10), VT (11), FF (12), CR (13), and space -(32), which are defined as white space in the "C" locale. This list may vary if -locale-specific matching is taking place. For example, in some locales the -"non-breaking space" character (\exA0) is recognized as white space, and in -others the VT character is not. -.P -A "word" character is an underscore or any character that is a letter or digit. -By default, the definition of letters and digits is controlled by PCRE's -low-valued character tables, and may vary if locale-specific matching is taking -place (see -.\" HTML <a href="pcreapi.html#localesupport"> -.\" </a> -"Locale support" -.\" -in the -.\" HREF -\fBpcreapi\fP -.\" -page). For example, in a French locale such as "fr_FR" in Unix-like systems, -or "french" in Windows, some character codes greater than 127 are used for -accented letters, and these are then matched by \ew. The use of locales with -Unicode is discouraged. -.P -By default, characters whose code points are greater than 127 never match \ed, -\es, or \ew, and always match \eD, \eS, and \eW, although this may vary for -characters in the range 128-255 when locale-specific matching is happening. -These escape sequences retain their original meanings from before Unicode -support was available, mainly for efficiency reasons. If PCRE is compiled with -Unicode property support, and the PCRE_UCP option is set, the behaviour is -changed so that Unicode properties are used to determine character types, as -follows: -.sp - \ed any character that matches \ep{Nd} (decimal digit) - \es any character that matches \ep{Z} or \eh or \ev - \ew any character that matches \ep{L} or \ep{N}, plus underscore -.sp -The upper case escapes match the inverse sets of characters. Note that \ed -matches only decimal digits, whereas \ew matches any Unicode digit, as well as -any Unicode letter, and underscore. Note also that PCRE_UCP affects \eb, and -\eB because they are defined in terms of \ew and \eW. Matching these sequences -is noticeably slower when PCRE_UCP is set. -.P -The sequences \eh, \eH, \ev, and \eV are features that were added to Perl at -release 5.10. In contrast to the other sequences, which match only ASCII -characters by default, these always match certain high-valued code points, -whether or not PCRE_UCP is set. The horizontal space characters are: -.sp - U+0009 Horizontal tab (HT) - U+0020 Space - U+00A0 Non-break space - U+1680 Ogham space mark - U+180E Mongolian vowel separator - U+2000 En quad - U+2001 Em quad - U+2002 En space - U+2003 Em space - U+2004 Three-per-em space - U+2005 Four-per-em space - U+2006 Six-per-em space - U+2007 Figure space - U+2008 Punctuation space - U+2009 Thin space - U+200A Hair space - U+202F Narrow no-break space - U+205F Medium mathematical space - U+3000 Ideographic space -.sp -The vertical space characters are: -.sp - U+000A Linefeed (LF) - U+000B Vertical tab (VT) - U+000C Form feed (FF) - U+000D Carriage return (CR) - U+0085 Next line (NEL) - U+2028 Line separator - U+2029 Paragraph separator -.sp -In 8-bit, non-UTF-8 mode, only the characters with codepoints less than 256 are -relevant. -. -. -.\" HTML <a name="newlineseq"></a> -.SS "Newline sequences" -.rs -.sp -Outside a character class, by default, the escape sequence \eR matches any -Unicode newline sequence. In 8-bit non-UTF-8 mode \eR is equivalent to the -following: -.sp - (?>\er\en|\en|\ex0b|\ef|\er|\ex85) -.sp -This is an example of an "atomic group", details of which are given -.\" HTML <a href="#atomicgroup"> -.\" </a> -below. -.\" -This particular group matches either the two-character sequence CR followed by -LF, or one of the single characters LF (linefeed, U+000A), VT (vertical tab, -U+000B), FF (form feed, U+000C), CR (carriage return, U+000D), or NEL (next -line, U+0085). The two-character sequence is treated as a single unit that -cannot be split. -.P -In other modes, two additional characters whose codepoints are greater than 255 -are added: LS (line separator, U+2028) and PS (paragraph separator, U+2029). -Unicode character property support is not needed for these characters to be -recognized. -.P -It is possible to restrict \eR to match only CR, LF, or CRLF (instead of the -complete set of Unicode line endings) by setting the option PCRE_BSR_ANYCRLF -either at compile time or when the pattern is matched. (BSR is an abbrevation -for "backslash R".) This can be made the default when PCRE is built; if this is -the case, the other behaviour can be requested via the PCRE_BSR_UNICODE option. -It is also possible to specify these settings by starting a pattern string with -one of the following sequences: -.sp - (*BSR_ANYCRLF) CR, LF, or CRLF only - (*BSR_UNICODE) any Unicode newline sequence -.sp -These override the default and the options given to the compiling function, but -they can themselves be overridden by options given to a matching function. Note -that these special settings, which are not Perl-compatible, are recognized only -at the very start of a pattern, and that they must be in upper case. If more -than one of them is present, the last one is used. They can be combined with a -change of newline convention; for example, a pattern can start with: -.sp - (*ANY)(*BSR_ANYCRLF) -.sp -They can also be combined with the (*UTF8), (*UTF16), (*UTF32), (*UTF) or -(*UCP) special sequences. Inside a character class, \eR is treated as an -unrecognized escape sequence, and so matches the letter "R" by default, but -causes an error if PCRE_EXTRA is set. -. -. -.\" HTML <a name="uniextseq"></a> -.SS Unicode character properties -.rs -.sp -When PCRE is built with Unicode character property support, three additional -escape sequences that match characters with specific properties are available. -When in 8-bit non-UTF-8 mode, these sequences are of course limited to testing -characters whose codepoints are less than 256, but they do work in this mode. -The extra escape sequences are: -.sp - \ep{\fIxx\fP} a character with the \fIxx\fP property - \eP{\fIxx\fP} a character without the \fIxx\fP property - \eX a Unicode extended grapheme cluster -.sp -The property names represented by \fIxx\fP above are limited to the Unicode -script names, the general category properties, "Any", which matches any -character (including newline), and some special PCRE properties (described -in the -.\" HTML <a href="#extraprops"> -.\" </a> -next section). -.\" -Other Perl properties such as "InMusicalSymbols" are not currently supported by -PCRE. Note that \eP{Any} does not match any characters, so always causes a -match failure. -.P -Sets of Unicode characters are defined as belonging to certain scripts. A -character from one of these sets can be matched using a script name. For -example: -.sp - \ep{Greek} - \eP{Han} -.sp -Those that are not part of an identified script are lumped together as -"Common". The current list of scripts is: -.P -Arabic, -Armenian, -Avestan, -Balinese, -Bamum, -Batak, -Bengali, -Bopomofo, -Brahmi, -Braille, -Buginese, -Buhid, -Canadian_Aboriginal, -Carian, -Chakma, -Cham, -Cherokee, -Common, -Coptic, -Cuneiform, -Cypriot, -Cyrillic, -Deseret, -Devanagari, -Egyptian_Hieroglyphs, -Ethiopic, -Georgian, -Glagolitic, -Gothic, -Greek, -Gujarati, -Gurmukhi, -Han, -Hangul, -Hanunoo, -Hebrew, -Hiragana, -Imperial_Aramaic, -Inherited, -Inscriptional_Pahlavi, -Inscriptional_Parthian, -Javanese, -Kaithi, -Kannada, -Katakana, -Kayah_Li, -Kharoshthi, -Khmer, -Lao, -Latin, -Lepcha, -Limbu, -Linear_B, -Lisu, -Lycian, -Lydian, -Malayalam, -Mandaic, -Meetei_Mayek, -Meroitic_Cursive, -Meroitic_Hieroglyphs, -Miao, -Mongolian, -Myanmar, -New_Tai_Lue, -Nko, -Ogham, -Old_Italic, -Old_Persian, -Old_South_Arabian, -Old_Turkic, -Ol_Chiki, -Oriya, -Osmanya, -Phags_Pa, -Phoenician, -Rejang, -Runic, -Samaritan, -Saurashtra, -Sharada, -Shavian, -Sinhala, -Sora_Sompeng, -Sundanese, -Syloti_Nagri, -Syriac, -Tagalog, -Tagbanwa, -Tai_Le, -Tai_Tham, -Tai_Viet, -Takri, -Tamil, -Telugu, -Thaana, -Thai, -Tibetan, -Tifinagh, -Ugaritic, -Vai, -Yi. -.P -Each character has exactly one Unicode general category property, specified by -a two-letter abbreviation. For compatibility with Perl, negation can be -specified by including a circumflex between the opening brace and the property -name. For example, \ep{^Lu} is the same as \eP{Lu}. -.P -If only one letter is specified with \ep or \eP, it includes all the general -category properties that start with that letter. In this case, in the absence -of negation, the curly brackets in the escape sequence are optional; these two -examples have the same effect: -.sp - \ep{L} - \epL -.sp -The following general category property codes are supported: -.sp - C Other - Cc Control - Cf Format - Cn Unassigned - Co Private use - Cs Surrogate -.sp - L Letter - Ll Lower case letter - Lm Modifier letter - Lo Other letter - Lt Title case letter - Lu Upper case letter -.sp - M Mark - Mc Spacing mark - Me Enclosing mark - Mn Non-spacing mark -.sp - N Number - Nd Decimal number - Nl Letter number - No Other number -.sp - P Punctuation - Pc Connector punctuation - Pd Dash punctuation - Pe Close punctuation - Pf Final punctuation - Pi Initial punctuation - Po Other punctuation - Ps Open punctuation -.sp - S Symbol - Sc Currency symbol - Sk Modifier symbol - Sm Mathematical symbol - So Other symbol -.sp - Z Separator - Zl Line separator - Zp Paragraph separator - Zs Space separator -.sp -The special property L& is also supported: it matches a character that has -the Lu, Ll, or Lt property, in other words, a letter that is not classified as -a modifier or "other". -.P -The Cs (Surrogate) property applies only to characters in the range U+D800 to -U+DFFF. Such characters are not valid in Unicode strings and so -cannot be tested by PCRE, unless UTF validity checking has been turned off -(see the discussion of PCRE_NO_UTF8_CHECK, PCRE_NO_UTF16_CHECK and -PCRE_NO_UTF32_CHECK in the -.\" HREF -\fBpcreapi\fP -.\" -page). Perl does not support the Cs property. -.P -The long synonyms for property names that Perl supports (such as \ep{Letter}) -are not supported by PCRE, nor is it permitted to prefix any of these -properties with "Is". -.P -No character that is in the Unicode table has the Cn (unassigned) property. -Instead, this property is assumed for any code point that is not in the -Unicode table. -.P -Specifying caseless matching does not affect these escape sequences. For -example, \ep{Lu} always matches only upper case letters. This is different from -the behaviour of current versions of Perl. -.P -Matching characters by Unicode property is not fast, because PCRE has to do a -multistage table lookup in order to find a character's property. That is why -the traditional escape sequences such as \ed and \ew do not use Unicode -properties in PCRE by default, though you can make them do so by setting the -PCRE_UCP option or by starting the pattern with (*UCP). -. -. -.SS Extended grapheme clusters -.rs -.sp -The \eX escape matches any number of Unicode characters that form an "extended -grapheme cluster", and treats the sequence as an atomic group -.\" HTML <a href="#atomicgroup"> -.\" </a> -(see below). -.\" -Up to and including release 8.31, PCRE matched an earlier, simpler definition -that was equivalent to -.sp - (?>\ePM\epM*) -.sp -That is, it matched a character without the "mark" property, followed by zero -or more characters with the "mark" property. Characters with the "mark" -property are typically non-spacing accents that affect the preceding character. -.P -This simple definition was extended in Unicode to include more complicated -kinds of composite character by giving each character a grapheme breaking -property, and creating rules that use these properties to define the boundaries -of extended grapheme clusters. In releases of PCRE later than 8.31, \eX matches -one of these clusters. -.P -\eX always matches at least one character. Then it decides whether to add -additional characters according to the following rules for ending a cluster: -.P -1. End at the end of the subject string. -.P -2. Do not end between CR and LF; otherwise end after any control character. -.P -3. Do not break Hangul (a Korean script) syllable sequences. Hangul characters -are of five types: L, V, T, LV, and LVT. An L character may be followed by an -L, V, LV, or LVT character; an LV or V character may be followed by a V or T -character; an LVT or T character may be follwed only by a T character. -.P -4. Do not end before extending characters or spacing marks. Characters with -the "mark" property always have the "extend" grapheme breaking property. -.P -5. Do not end after prepend characters. -.P -6. Otherwise, end the cluster. -. -. -.\" HTML <a name="extraprops"></a> -.SS PCRE's additional properties -.rs -.sp -As well as the standard Unicode properties described above, PCRE supports four -more that make it possible to convert traditional escape sequences such as \ew -and \es to use Unicode properties. PCRE uses these non-standard, non-Perl -properties internally when PCRE_UCP is set. However, they may also be used -explicitly. These properties are: -.sp - Xan Any alphanumeric character - Xps Any POSIX space character - Xsp Any Perl space character - Xwd Any Perl "word" character -.sp -Xan matches characters that have either the L (letter) or the N (number) -property. Xps matches the characters tab, linefeed, vertical tab, form feed, or -carriage return, and any other character that has the Z (separator) property. -Xsp is the same as Xps; it used to exclude vertical tab, for Perl -compatibility, but Perl changed, and so PCRE followed at release 8.34. Xwd -matches the same characters as Xan, plus underscore. -.P -There is another non-standard property, Xuc, which matches any character that -can be represented by a Universal Character Name in C++ and other programming -languages. These are the characters $, @, ` (grave accent), and all characters -with Unicode code points greater than or equal to U+00A0, except for the -surrogates U+D800 to U+DFFF. Note that most base (ASCII) characters are -excluded. (Universal Character Names are of the form \euHHHH or \eUHHHHHHHH -where H is a hexadecimal digit. Note that the Xuc property does not match these -sequences but the characters that they represent.) -. -. -.\" HTML <a name="resetmatchstart"></a> -.SS "Resetting the match start" -.rs -.sp -The escape sequence \eK causes any previously matched characters not to be -included in the final matched sequence. For example, the pattern: -.sp - foo\eKbar -.sp -matches "foobar", but reports that it has matched "bar". This feature is -similar to a lookbehind assertion -.\" HTML <a href="#lookbehind"> -.\" </a> -(described below). -.\" -However, in this case, the part of the subject before the real match does not -have to be of fixed length, as lookbehind assertions do. The use of \eK does -not interfere with the setting of -.\" HTML <a href="#subpattern"> -.\" </a> -captured substrings. -.\" -For example, when the pattern -.sp - (foo)\eKbar -.sp -matches "foobar", the first substring is still set to "foo". -.P -Perl documents that the use of \eK within assertions is "not well defined". In -PCRE, \eK is acted upon when it occurs inside positive assertions, but is -ignored in negative assertions. Note that when a pattern such as (?=ab\eK) -matches, the reported start of the match can be greater than the end of the -match. -. -. -.\" HTML <a name="smallassertions"></a> -.SS "Simple assertions" -.rs -.sp -The final use of backslash is for certain simple assertions. An assertion -specifies a condition that has to be met at a particular point in a match, -without consuming any characters from the subject string. The use of -subpatterns for more complicated assertions is described -.\" HTML <a href="#bigassertions"> -.\" </a> -below. -.\" -The backslashed assertions are: -.sp - \eb matches at a word boundary - \eB matches when not at a word boundary - \eA matches at the start of the subject - \eZ matches at the end of the subject - also matches before a newline at the end of the subject - \ez matches only at the end of the subject - \eG matches at the first matching position in the subject -.sp -Inside a character class, \eb has a different meaning; it matches the backspace -character. If any other of these assertions appears in a character class, by -default it matches the corresponding literal character (for example, \eB -matches the letter B). However, if the PCRE_EXTRA option is set, an "invalid -escape sequence" error is generated instead. -.P -A word boundary is a position in the subject string where the current character -and the previous character do not both match \ew or \eW (i.e. one matches -\ew and the other matches \eW), or the start or end of the string if the -first or last character matches \ew, respectively. In a UTF mode, the meanings -of \ew and \eW can be changed by setting the PCRE_UCP option. When this is -done, it also affects \eb and \eB. Neither PCRE nor Perl has a separate "start -of word" or "end of word" metasequence. However, whatever follows \eb normally -determines which it is. For example, the fragment \eba matches "a" at the start -of a word. -.P -The \eA, \eZ, and \ez assertions differ from the traditional circumflex and -dollar (described in the next section) in that they only ever match at the very -start and end of the subject string, whatever options are set. Thus, they are -independent of multiline mode. These three assertions are not affected by the -PCRE_NOTBOL or PCRE_NOTEOL options, which affect only the behaviour of the -circumflex and dollar metacharacters. However, if the \fIstartoffset\fP -argument of \fBpcre_exec()\fP is non-zero, indicating that matching is to start -at a point other than the beginning of the subject, \eA can never match. The -difference between \eZ and \ez is that \eZ matches before a newline at the end -of the string as well as at the very end, whereas \ez matches only at the end. -.P -The \eG assertion is true only when the current matching position is at the -start point of the match, as specified by the \fIstartoffset\fP argument of -\fBpcre_exec()\fP. It differs from \eA when the value of \fIstartoffset\fP is -non-zero. By calling \fBpcre_exec()\fP multiple times with appropriate -arguments, you can mimic Perl's /g option, and it is in this kind of -implementation where \eG can be useful. -.P -Note, however, that PCRE's interpretation of \eG, as the start of the current -match, is subtly different from Perl's, which defines it as the end of the -previous match. In Perl, these can be different when the previously matched -string was empty. Because PCRE does just one match at a time, it cannot -reproduce this behaviour. -.P -If all the alternatives of a pattern begin with \eG, the expression is anchored -to the starting match position, and the "anchored" flag is set in the compiled -regular expression. -. -. -.SH "CIRCUMFLEX AND DOLLAR" -.rs -.sp -The circumflex and dollar metacharacters are zero-width assertions. That is, -they test for a particular condition being true without consuming any -characters from the subject string. -.P -Outside a character class, in the default matching mode, the circumflex -character is an assertion that is true only if the current matching point is at -the start of the subject string. If the \fIstartoffset\fP argument of -\fBpcre_exec()\fP is non-zero, circumflex can never match if the PCRE_MULTILINE -option is unset. Inside a character class, circumflex has an entirely different -meaning -.\" HTML <a href="#characterclass"> -.\" </a> -(see below). -.\" -.P -Circumflex need not be the first character of the pattern if a number of -alternatives are involved, but it should be the first thing in each alternative -in which it appears if the pattern is ever to match that branch. If all -possible alternatives start with a circumflex, that is, if the pattern is -constrained to match only at the start of the subject, it is said to be an -"anchored" pattern. (There are also other constructs that can cause a pattern -to be anchored.) -.P -The dollar character is an assertion that is true only if the current matching -point is at the end of the subject string, or immediately before a newline at -the end of the string (by default). Note, however, that it does not actually -match the newline. Dollar need not be the last character of the pattern if a -number of alternatives are involved, but it should be the last item in any -branch in which it appears. Dollar has no special meaning in a character class. -.P -The meaning of dollar can be changed so that it matches only at the very end of -the string, by setting the PCRE_DOLLAR_ENDONLY option at compile time. This -does not affect the \eZ assertion. -.P -The meanings of the circumflex and dollar characters are changed if the -PCRE_MULTILINE option is set. When this is the case, a circumflex matches -immediately after internal newlines as well as at the start of the subject -string. It does not match after a newline that ends the string. A dollar -matches before any newlines in the string, as well as at the very end, when -PCRE_MULTILINE is set. When newline is specified as the two-character -sequence CRLF, isolated CR and LF characters do not indicate newlines. -.P -For example, the pattern /^abc$/ matches the subject string "def\enabc" (where -\en represents a newline) in multiline mode, but not otherwise. Consequently, -patterns that are anchored in single line mode because all branches start with -^ are not anchored in multiline mode, and a match for circumflex is possible -when the \fIstartoffset\fP argument of \fBpcre_exec()\fP is non-zero. The -PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is set. -.P -Note that the sequences \eA, \eZ, and \ez can be used to match the start and -end of the subject in both modes, and if all branches of a pattern start with -\eA it is always anchored, whether or not PCRE_MULTILINE is set. -. -. -.\" HTML <a name="fullstopdot"></a> -.SH "FULL STOP (PERIOD, DOT) AND \eN" -.rs -.sp -Outside a character class, a dot in the pattern matches any one character in -the subject string except (by default) a character that signifies the end of a -line. -.P -When a line ending is defined as a single character, dot never matches that -character; when the two-character sequence CRLF is used, dot does not match CR -if it is immediately followed by LF, but otherwise it matches all characters -(including isolated CRs and LFs). When any Unicode line endings are being -recognized, dot does not match CR or LF or any of the other line ending -characters. -.P -The behaviour of dot with regard to newlines can be changed. If the PCRE_DOTALL -option is set, a dot matches any one character, without exception. If the -two-character sequence CRLF is present in the subject string, it takes two dots -to match it. -.P -The handling of dot is entirely independent of the handling of circumflex and -dollar, the only relationship being that they both involve newlines. Dot has no -special meaning in a character class. -.P -The escape sequence \eN behaves like a dot, except that it is not affected by -the PCRE_DOTALL option. In other words, it matches any character except one -that signifies the end of a line. Perl also uses \eN to match characters by -name; PCRE does not support this. -. -. -.SH "MATCHING A SINGLE DATA UNIT" -.rs -.sp -Outside a character class, the escape sequence \eC matches any one data unit, -whether or not a UTF mode is set. In the 8-bit library, one data unit is one -byte; in the 16-bit library it is a 16-bit unit; in the 32-bit library it is -a 32-bit unit. Unlike a dot, \eC always -matches line-ending characters. The feature is provided in Perl in order to -match individual bytes in UTF-8 mode, but it is unclear how it can usefully be -used. Because \eC breaks up characters into individual data units, matching one -unit with \eC in a UTF mode means that the rest of the string may start with a -malformed UTF character. This has undefined results, because PCRE assumes that -it is dealing with valid UTF strings (and by default it checks this at the -start of processing unless the PCRE_NO_UTF8_CHECK, PCRE_NO_UTF16_CHECK or -PCRE_NO_UTF32_CHECK option is used). -.P -PCRE does not allow \eC to appear in lookbehind assertions -.\" HTML <a href="#lookbehind"> -.\" </a> -(described below) -.\" -in a UTF mode, because this would make it impossible to calculate the length of -the lookbehind. -.P -In general, the \eC escape sequence is best avoided. However, one -way of using it that avoids the problem of malformed UTF characters is to use a -lookahead to check the length of the next character, as in this pattern, which -could be used with a UTF-8 string (ignore white space and line breaks): -.sp - (?| (?=[\ex00-\ex7f])(\eC) | - (?=[\ex80-\ex{7ff}])(\eC)(\eC) | - (?=[\ex{800}-\ex{ffff}])(\eC)(\eC)(\eC) | - (?=[\ex{10000}-\ex{1fffff}])(\eC)(\eC)(\eC)(\eC)) -.sp -A group that starts with (?| resets the capturing parentheses numbers in each -alternative (see -.\" HTML <a href="#dupsubpatternnumber"> -.\" </a> -"Duplicate Subpattern Numbers" -.\" -below). The assertions at the start of each branch check the next UTF-8 -character for values whose encoding uses 1, 2, 3, or 4 bytes, respectively. The -character's individual bytes are then captured by the appropriate number of -groups. -. -. -.\" HTML <a name="characterclass"></a> -.SH "SQUARE BRACKETS AND CHARACTER CLASSES" -.rs -.sp -An opening square bracket introduces a character class, terminated by a closing -square bracket. A closing square bracket on its own is not special by default. -However, if the PCRE_JAVASCRIPT_COMPAT option is set, a lone closing square -bracket causes a compile-time error. If a closing square bracket is required as -a member of the class, it should be the first data character in the class -(after an initial circumflex, if present) or escaped with a backslash. -.P -A character class matches a single character in the subject. In a UTF mode, the -character may be more than one data unit long. A matched character must be in -the set of characters defined by the class, unless the first character in the -class definition is a circumflex, in which case the subject character must not -be in the set defined by the class. If a circumflex is actually required as a -member of the class, ensure it is not the first character, or escape it with a -backslash. -.P -For example, the character class [aeiou] matches any lower case vowel, while -[^aeiou] matches any character that is not a lower case vowel. Note that a -circumflex is just a convenient notation for specifying the characters that -are in the class by enumerating those that are not. A class that starts with a -circumflex is not an assertion; it still consumes a character from the subject -string, and therefore it fails if the current pointer is at the end of the -string. -.P -In UTF-8 (UTF-16, UTF-32) mode, characters with values greater than 255 (0xffff) -can be included in a class as a literal string of data units, or by using the -\ex{ escaping mechanism. -.P -When caseless matching is set, any letters in a class represent both their -upper case and lower case versions, so for example, a caseless [aeiou] matches -"A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a -caseful version would. In a UTF mode, PCRE always understands the concept of -case for characters whose values are less than 128, so caseless matching is -always possible. For characters with higher values, the concept of case is -supported if PCRE is compiled with Unicode property support, but not otherwise. -If you want to use caseless matching in a UTF mode for characters 128 and -above, you must ensure that PCRE is compiled with Unicode property support as -well as with UTF support. -.P -Characters that might indicate line breaks are never treated in any special way -when matching character classes, whatever line-ending sequence is in use, and -whatever setting of the PCRE_DOTALL and PCRE_MULTILINE options is used. A class -such as [^a] always matches one of these characters. -.P -The minus (hyphen) character can be used to specify a range of characters in a -character class. For example, [d-m] matches any letter between d and m, -inclusive. If a minus character is required in a class, it must be escaped with -a backslash or appear in a position where it cannot be interpreted as -indicating a range, typically as the first or last character in the class, or -immediately after a range. For example, [b-d-z] matches letters in the range b -to d, a hyphen character, or z. -.P -It is not possible to have the literal character "]" as the end character of a -range. A pattern such as [W-]46] is interpreted as a class of two characters -("W" and "-") followed by a literal string "46]", so it would match "W46]" or -"-46]". However, if the "]" is escaped with a backslash it is interpreted as -the end of range, so [W-\e]46] is interpreted as a class containing a range -followed by two other characters. The octal or hexadecimal representation of -"]" can also be used to end a range. -.P -An error is generated if a POSIX character class (see below) or an escape -sequence other than one that defines a single character appears at a point -where a range ending character is expected. For example, [z-\exff] is valid, -but [A-\ed] and [A-[:digit:]] are not. -.P -Ranges operate in the collating sequence of character values. They can also be -used for characters specified numerically, for example [\e000-\e037]. Ranges -can include any characters that are valid for the current mode. -.P -If a range that includes letters is used when caseless matching is set, it -matches the letters in either case. For example, [W-c] is equivalent to -[][\e\e^_`wxyzabc], matched caselessly, and in a non-UTF mode, if character -tables for a French locale are in use, [\exc8-\excb] matches accented E -characters in both cases. In UTF modes, PCRE supports the concept of case for -characters with values greater than 128 only when it is compiled with Unicode -property support. -.P -The character escape sequences \ed, \eD, \eh, \eH, \ep, \eP, \es, \eS, \ev, -\eV, \ew, and \eW may appear in a character class, and add the characters that -they match to the class. For example, [\edABCDEF] matches any hexadecimal -digit. In UTF modes, the PCRE_UCP option affects the meanings of \ed, \es, \ew -and their upper case partners, just as it does when they appear outside a -character class, as described in the section entitled -.\" HTML <a href="#genericchartypes"> -.\" </a> -"Generic character types" -.\" -above. The escape sequence \eb has a different meaning inside a character -class; it matches the backspace character. The sequences \eB, \eN, \eR, and \eX -are not special inside a character class. Like any other unrecognized escape -sequences, they are treated as the literal characters "B", "N", "R", and "X" by -default, but cause an error if the PCRE_EXTRA option is set. -.P -A circumflex can conveniently be used with the upper case character types to -specify a more restricted set of characters than the matching lower case type. -For example, the class [^\eW_] matches any letter or digit, but not underscore, -whereas [\ew] includes underscore. A positive character class should be read as -"something OR something OR ..." and a negative class as "NOT something AND NOT -something AND NOT ...". -.P -The only metacharacters that are recognized in character classes are backslash, -hyphen (only where it can be interpreted as specifying a range), circumflex -(only at the start), opening square bracket (only when it can be interpreted as -introducing a POSIX class name, or for a special compatibility feature - see -the next two sections), and the terminating closing square bracket. However, -escaping other non-alphanumeric characters does no harm. -. -. -.SH "POSIX CHARACTER CLASSES" -.rs -.sp -Perl supports the POSIX notation for character classes. This uses names -enclosed by [: and :] within the enclosing square brackets. PCRE also supports -this notation. For example, -.sp - [01[:alpha:]%] -.sp -matches "0", "1", any alphabetic character, or "%". The supported class names -are: -.sp - alnum letters and digits - alpha letters - ascii character codes 0 - 127 - blank space or tab only - cntrl control characters - digit decimal digits (same as \ed) - graph printing characters, excluding space - lower lower case letters - print printing characters, including space - punct printing characters, excluding letters and digits and space - space white space (the same as \es from PCRE 8.34) - upper upper case letters - word "word" characters (same as \ew) - xdigit hexadecimal digits -.sp -The default "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), -and space (32). If locale-specific matching is taking place, the list of space -characters may be different; there may be fewer or more of them. "Space" used -to be different to \es, which did not include VT, for Perl compatibility. -However, Perl changed at release 5.18, and PCRE followed at release 8.34. -"Space" and \es now match the same set of characters. -.P -The name "word" is a Perl extension, and "blank" is a GNU extension from Perl -5.8. Another Perl extension is negation, which is indicated by a ^ character -after the colon. For example, -.sp - [12[:^digit:]] -.sp -matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the POSIX -syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not -supported, and an error is given if they are encountered. -.P -By default, characters with values greater than 128 do not match any of the -POSIX character classes. However, if the PCRE_UCP option is passed to -\fBpcre_compile()\fP, some of the classes are changed so that Unicode character -properties are used. This is achieved by replacing certain POSIX classes by -other sequences, as follows: -.sp - [:alnum:] becomes \ep{Xan} - [:alpha:] becomes \ep{L} - [:blank:] becomes \eh - [:digit:] becomes \ep{Nd} - [:lower:] becomes \ep{Ll} - [:space:] becomes \ep{Xps} - [:upper:] becomes \ep{Lu} - [:word:] becomes \ep{Xwd} -.sp -Negated versions, such as [:^alpha:] use \eP instead of \ep. Three other POSIX -classes are handled specially in UCP mode: -.TP 10 -[:graph:] -This matches characters that have glyphs that mark the page when printed. In -Unicode property terms, it matches all characters with the L, M, N, P, S, or Cf -properties, except for: -.sp - U+061C Arabic Letter Mark - U+180E Mongolian Vowel Separator - U+2066 - U+2069 Various "isolate"s -.sp -.TP 10 -[:print:] -This matches the same characters as [:graph:] plus space characters that are -not controls, that is, characters with the Zs property. -.TP 10 -[:punct:] -This matches all characters that have the Unicode P (punctuation) property, -plus those characters whose code points are less than 128 that have the S -(Symbol) property. -.P -The other POSIX classes are unchanged, and match only characters with code -points less than 128. -. -. -.SH "COMPATIBILITY FEATURE FOR WORD BOUNDARIES" -.rs -.sp -In the POSIX.2 compliant library that was included in 4.4BSD Unix, the ugly -syntax [[:<:]] and [[:>:]] is used for matching "start of word" and "end of -word". PCRE treats these items as follows: -.sp - [[:<:]] is converted to \eb(?=\ew) - [[:>:]] is converted to \eb(?<=\ew) -.sp -Only these exact character sequences are recognized. A sequence such as -[a[:<:]b] provokes error for an unrecognized POSIX class name. This support is -not compatible with Perl. It is provided to help migrations from other -environments, and is best not used in any new patterns. Note that \eb matches -at the start and the end of a word (see -.\" HTML <a href="#smallassertions"> -.\" </a> -"Simple assertions" -.\" -above), and in a Perl-style pattern the preceding or following character -normally shows which is wanted, without the need for the assertions that are -used above in order to give exactly the POSIX behaviour. -. -. -.SH "VERTICAL BAR" -.rs -.sp -Vertical bar characters are used to separate alternative patterns. For example, -the pattern -.sp - gilbert|sullivan -.sp -matches either "gilbert" or "sullivan". Any number of alternatives may appear, -and an empty alternative is permitted (matching the empty string). The matching -process tries each alternative in turn, from left to right, and the first one -that succeeds is used. If the alternatives are within a subpattern -.\" HTML <a href="#subpattern"> -.\" </a> -(defined below), -.\" -"succeeds" means matching the rest of the main pattern as well as the -alternative in the subpattern. -. -. -.SH "INTERNAL OPTION SETTING" -.rs -.sp -The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and -PCRE_EXTENDED options (which are Perl-compatible) can be changed from within -the pattern by a sequence of Perl option letters enclosed between "(?" and ")". -The option letters are -.sp - i for PCRE_CASELESS - m for PCRE_MULTILINE - s for PCRE_DOTALL - x for PCRE_EXTENDED -.sp -For example, (?im) sets caseless, multiline matching. It is also possible to -unset these options by preceding the letter with a hyphen, and a combined -setting and unsetting such as (?im-sx), which sets PCRE_CASELESS and -PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, is also -permitted. If a letter appears both before and after the hyphen, the option is -unset. -.P -The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA can be -changed in the same way as the Perl-compatible options by using the characters -J, U and X respectively. -.P -When one of these option changes occurs at top level (that is, not inside -subpattern parentheses), the change applies to the remainder of the pattern -that follows. If the change is placed right at the start of a pattern, PCRE -extracts it into the global options (and it will therefore show up in data -extracted by the \fBpcre_fullinfo()\fP function). -.P -An option change within a subpattern (see below for a description of -subpatterns) affects only that part of the subpattern that follows it, so -.sp - (a(?i)b)c -.sp -matches abc and aBc and no other strings (assuming PCRE_CASELESS is not used). -By this means, options can be made to have different settings in different -parts of the pattern. Any changes made in one alternative do carry on -into subsequent branches within the same subpattern. For example, -.sp - (a(?i)b|c) -.sp -matches "ab", "aB", "c", and "C", even though when matching "C" the first -branch is abandoned before the option setting. This is because the effects of -option settings happen at compile time. There would be some very weird -behaviour otherwise. -.P -\fBNote:\fP There are other PCRE-specific options that can be set by the -application when the compiling or matching functions are called. In some cases -the pattern can contain special leading sequences such as (*CRLF) to override -what the application has set or what has been defaulted. Details are given in -the section entitled -.\" HTML <a href="#newlineseq"> -.\" </a> -"Newline sequences" -.\" -above. There are also the (*UTF8), (*UTF16),(*UTF32), and (*UCP) leading -sequences that can be used to set UTF and Unicode property modes; they are -equivalent to setting the PCRE_UTF8, PCRE_UTF16, PCRE_UTF32 and the PCRE_UCP -options, respectively. The (*UTF) sequence is a generic version that can be -used with any of the libraries. However, the application can set the -PCRE_NEVER_UTF option, which locks out the use of the (*UTF) sequences. -. -. -.\" HTML <a name="subpattern"></a> -.SH SUBPATTERNS -.rs -.sp -Subpatterns are delimited by parentheses (round brackets), which can be nested. -Turning part of a pattern into a subpattern does two things: -.sp -1. It localizes a set of alternatives. For example, the pattern -.sp - cat(aract|erpillar|) -.sp -matches "cataract", "caterpillar", or "cat". Without the parentheses, it would -match "cataract", "erpillar" or an empty string. -.sp -2. It sets up the subpattern as a capturing subpattern. This means that, when -the whole pattern matches, that portion of the subject string that matched the -subpattern is passed back to the caller via the \fIovector\fP argument of the -matching function. (This applies only to the traditional matching functions; -the DFA matching functions do not support capturing.) -.P -Opening parentheses are counted from left to right (starting from 1) to obtain -numbers for the capturing subpatterns. For example, if the string "the red -king" is matched against the pattern -.sp - the ((red|white) (king|queen)) -.sp -the captured substrings are "red king", "red", and "king", and are numbered 1, -2, and 3, respectively. -.P -The fact that plain parentheses fulfil two functions is not always helpful. -There are often times when a grouping subpattern is required without a -capturing requirement. If an opening parenthesis is followed by a question mark -and a colon, the subpattern does not do any capturing, and is not counted when -computing the number of any subsequent capturing subpatterns. For example, if -the string "the white queen" is matched against the pattern -.sp - the ((?:red|white) (king|queen)) -.sp -the captured substrings are "white queen" and "queen", and are numbered 1 and -2. The maximum number of capturing subpatterns is 65535. -.P -As a convenient shorthand, if any option settings are required at the start of -a non-capturing subpattern, the option letters may appear between the "?" and -the ":". Thus the two patterns -.sp - (?i:saturday|sunday) - (?:(?i)saturday|sunday) -.sp -match exactly the same set of strings. Because alternative branches are tried -from left to right, and options are not reset until the end of the subpattern -is reached, an option setting in one branch does affect subsequent branches, so -the above patterns match "SUNDAY" as well as "Saturday". -. -. -.\" HTML <a name="dupsubpatternnumber"></a> -.SH "DUPLICATE SUBPATTERN NUMBERS" -.rs -.sp -Perl 5.10 introduced a feature whereby each alternative in a subpattern uses -the same numbers for its capturing parentheses. Such a subpattern starts with -(?| and is itself a non-capturing subpattern. For example, consider this -pattern: -.sp - (?|(Sat)ur|(Sun))day -.sp -Because the two alternatives are inside a (?| group, both sets of capturing -parentheses are numbered one. Thus, when the pattern matches, you can look -at captured substring number one, whichever alternative matched. This construct -is useful when you want to capture part, but not all, of one of a number of -alternatives. Inside a (?| group, parentheses are numbered as usual, but the -number is reset at the start of each branch. The numbers of any capturing -parentheses that follow the subpattern start after the highest number used in -any branch. The following example is taken from the Perl documentation. The -numbers underneath show in which buffer the captured content will be stored. -.sp - # before ---------------branch-reset----------- after - / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x - # 1 2 2 3 2 3 4 -.sp -A back reference to a numbered subpattern uses the most recent value that is -set for that number by any subpattern. The following pattern matches "abcabc" -or "defdef": -.sp - /(?|(abc)|(def))\e1/ -.sp -In contrast, a subroutine call to a numbered subpattern always refers to the -first one in the pattern with the given number. The following pattern matches -"abcabc" or "defabc": -.sp - /(?|(abc)|(def))(?1)/ -.sp -If a -.\" HTML <a href="#conditions"> -.\" </a> -condition test -.\" -for a subpattern's having matched refers to a non-unique number, the test is -true if any of the subpatterns of that number have matched. -.P -An alternative approach to using this "branch reset" feature is to use -duplicate named subpatterns, as described in the next section. -. -. -.SH "NAMED SUBPATTERNS" -.rs -.sp -Identifying capturing parentheses by number is simple, but it can be very hard -to keep track of the numbers in complicated regular expressions. Furthermore, -if an expression is modified, the numbers may change. To help with this -difficulty, PCRE supports the naming of subpatterns. This feature was not -added to Perl until release 5.10. Python had the feature earlier, and PCRE -introduced it at release 4.0, using the Python syntax. PCRE now supports both -the Perl and the Python syntax. Perl allows identically numbered subpatterns to -have different names, but PCRE does not. -.P -In PCRE, a subpattern can be named in one of three ways: (?<name>...) or -(?'name'...) as in Perl, or (?P<name>...) as in Python. References to capturing -parentheses from other parts of the pattern, such as -.\" HTML <a href="#backreferences"> -.\" </a> -back references, -.\" -.\" HTML <a href="#recursion"> -.\" </a> -recursion, -.\" -and -.\" HTML <a href="#conditions"> -.\" </a> -conditions, -.\" -can be made by name as well as by number. -.P -Names consist of up to 32 alphanumeric characters and underscores, but must -start with a non-digit. Named capturing parentheses are still allocated numbers -as well as names, exactly as if the names were not present. The PCRE API -provides function calls for extracting the name-to-number translation table -from a compiled pattern. There is also a convenience function for extracting a -captured substring by name. -.P -By default, a name must be unique within a pattern, but it is possible to relax -this constraint by setting the PCRE_DUPNAMES option at compile time. (Duplicate -names are also always permitted for subpatterns with the same number, set up as -described in the previous section.) Duplicate names can be useful for patterns -where only one instance of the named parentheses can match. Suppose you want to -match the name of a weekday, either as a 3-letter abbreviation or as the full -name, and in both cases you want to extract the abbreviation. This pattern -(ignoring the line breaks) does the job: -.sp - (?<DN>Mon|Fri|Sun)(?:day)?| - (?<DN>Tue)(?:sday)?| - (?<DN>Wed)(?:nesday)?| - (?<DN>Thu)(?:rsday)?| - (?<DN>Sat)(?:urday)? -.sp -There are five capturing substrings, but only one is ever set after a match. -(An alternative way of solving this problem is to use a "branch reset" -subpattern, as described in the previous section.) -.P -The convenience function for extracting the data by name returns the substring -for the first (and in this example, the only) subpattern of that name that -matched. This saves searching to find which numbered subpattern it was. -.P -If you make a back reference to a non-unique named subpattern from elsewhere in -the pattern, the subpatterns to which the name refers are checked in the order -in which they appear in the overall pattern. The first one that is set is used -for the reference. For example, this pattern matches both "foofoo" and -"barbar" but not "foobar" or "barfoo": -.sp - (?:(?<n>foo)|(?<n>bar))\ek<n> -.sp -.P -If you make a subroutine call to a non-unique named subpattern, the one that -corresponds to the first occurrence of the name is used. In the absence of -duplicate numbers (see the previous section) this is the one with the lowest -number. -.P -If you use a named reference in a condition -test (see the -.\" -.\" HTML <a href="#conditions"> -.\" </a> -section about conditions -.\" -below), either to check whether a subpattern has matched, or to check for -recursion, all subpatterns with the same name are tested. If the condition is -true for any one of them, the overall condition is true. This is the same -behaviour as testing by number. For further details of the interfaces for -handling named subpatterns, see the -.\" HREF -\fBpcreapi\fP -.\" -documentation. -.P -\fBWarning:\fP You cannot use different names to distinguish between two -subpatterns with the same number because PCRE uses only the numbers when -matching. For this reason, an error is given at compile time if different names -are given to subpatterns with the same number. However, you can always give the -same name to subpatterns with the same number, even when PCRE_DUPNAMES is not -set. -. -. -.SH REPETITION -.rs -.sp -Repetition is specified by quantifiers, which can follow any of the following -items: -.sp - a literal data character - the dot metacharacter - the \eC escape sequence - the \eX escape sequence - the \eR escape sequence - an escape such as \ed or \epL that matches a single character - a character class - a back reference (see next section) - a parenthesized subpattern (including assertions) - a subroutine call to a subpattern (recursive or otherwise) -.sp -The general repetition quantifier specifies a minimum and maximum number of -permitted matches, by giving the two numbers in curly brackets (braces), -separated by a comma. The numbers must be less than 65536, and the first must -be less than or equal to the second. For example: -.sp - z{2,4} -.sp -matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special -character. If the second number is omitted, but the comma is present, there is -no upper limit; if the second number and the comma are both omitted, the -quantifier specifies an exact number of required matches. Thus -.sp - [aeiou]{3,} -.sp -matches at least 3 successive vowels, but may match many more, while -.sp - \ed{8} -.sp -matches exactly 8 digits. An opening curly bracket that appears in a position -where a quantifier is not allowed, or one that does not match the syntax of a -quantifier, is taken as a literal character. For example, {,6} is not a -quantifier, but a literal string of four characters. -.P -In UTF modes, quantifiers apply to characters rather than to individual data -units. Thus, for example, \ex{100}{2} matches two characters, each of -which is represented by a two-byte sequence in a UTF-8 string. Similarly, -\eX{3} matches three Unicode extended grapheme clusters, each of which may be -several data units long (and they may be of different lengths). -.P -The quantifier {0} is permitted, causing the expression to behave as if the -previous item and the quantifier were not present. This may be useful for -subpatterns that are referenced as -.\" HTML <a href="#subpatternsassubroutines"> -.\" </a> -subroutines -.\" -from elsewhere in the pattern (but see also the section entitled -.\" HTML <a href="#subdefine"> -.\" </a> -"Defining subpatterns for use by reference only" -.\" -below). Items other than subpatterns that have a {0} quantifier are omitted -from the compiled pattern. -.P -For convenience, the three most common quantifiers have single-character -abbreviations: -.sp - * is equivalent to {0,} - + is equivalent to {1,} - ? is equivalent to {0,1} -.sp -It is possible to construct infinite loops by following a subpattern that can -match no characters with a quantifier that has no upper limit, for example: -.sp - (a?)* -.sp -Earlier versions of Perl and PCRE used to give an error at compile time for -such patterns. However, because there are cases where this can be useful, such -patterns are now accepted, but if any repetition of the subpattern does in fact -match no characters, the loop is forcibly broken. -.P -By default, the quantifiers are "greedy", that is, they match as much as -possible (up to the maximum number of permitted times), without causing the -rest of the pattern to fail. The classic example of where this gives problems -is in trying to match comments in C programs. These appear between /* and */ -and within the comment, individual * and / characters may appear. An attempt to -match C comments by applying the pattern -.sp - /\e*.*\e*/ -.sp -to the string -.sp - /* first comment */ not comment /* second comment */ -.sp -fails, because it matches the entire string owing to the greediness of the .* -item. -.P -However, if a quantifier is followed by a question mark, it ceases to be -greedy, and instead matches the minimum number of times possible, so the -pattern -.sp - /\e*.*?\e*/ -.sp -does the right thing with the C comments. The meaning of the various -quantifiers is not otherwise changed, just the preferred number of matches. -Do not confuse this use of question mark with its use as a quantifier in its -own right. Because it has two uses, it can sometimes appear doubled, as in -.sp - \ed??\ed -.sp -which matches one digit by preference, but can match two if that is the only -way the rest of the pattern matches. -.P -If the PCRE_UNGREEDY option is set (an option that is not available in Perl), -the quantifiers are not greedy by default, but individual ones can be made -greedy by following them with a question mark. In other words, it inverts the -default behaviour. -.P -When a parenthesized subpattern is quantified with a minimum repeat count that -is greater than 1 or with a limited maximum, more memory is required for the -compiled pattern, in proportion to the size of the minimum or maximum. -.P -If a pattern starts with .* or .{0,} and the PCRE_DOTALL option (equivalent -to Perl's /s) is set, thus allowing the dot to match newlines, the pattern is -implicitly anchored, because whatever follows will be tried against every -character position in the subject string, so there is no point in retrying the -overall match at any position after the first. PCRE normally treats such a -pattern as though it were preceded by \eA. -.P -In cases where it is known that the subject string contains no newlines, it is -worth setting PCRE_DOTALL in order to obtain this optimization, or -alternatively using ^ to indicate anchoring explicitly. -.P -However, there are some cases where the optimization cannot be used. When .* -is inside capturing parentheses that are the subject of a back reference -elsewhere in the pattern, a match at the start may fail where a later one -succeeds. Consider, for example: -.sp - (.*)abc\e1 -.sp -If the subject is "xyz123abc123" the match point is the fourth character. For -this reason, such a pattern is not implicitly anchored. -.P -Another case where implicit anchoring is not applied is when the leading .* is -inside an atomic group. Once again, a match at the start may fail where a later -one succeeds. Consider this pattern: -.sp - (?>.*?a)b -.sp -It matches "ab" in the subject "aab". The use of the backtracking control verbs -(*PRUNE) and (*SKIP) also disable this optimization. -.P -When a capturing subpattern is repeated, the value captured is the substring -that matched the final iteration. For example, after -.sp - (tweedle[dume]{3}\es*)+ -.sp -has matched "tweedledum tweedledee" the value of the captured substring is -"tweedledee". However, if there are nested capturing subpatterns, the -corresponding captured values may have been set in previous iterations. For -example, after -.sp - /(a|(b))+/ -.sp -matches "aba" the value of the second captured substring is "b". -. -. -.\" HTML <a name="atomicgroup"></a> -.SH "ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS" -.rs -.sp -With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") -repetition, failure of what follows normally causes the repeated item to be -re-evaluated to see if a different number of repeats allows the rest of the -pattern to match. Sometimes it is useful to prevent this, either to change the -nature of the match, or to cause it fail earlier than it otherwise might, when -the author of the pattern knows there is no point in carrying on. -.P -Consider, for example, the pattern \ed+foo when applied to the subject line -.sp - 123456bar -.sp -After matching all 6 digits and then failing to match "foo", the normal -action of the matcher is to try again with only 5 digits matching the \ed+ -item, and then with 4, and so on, before ultimately failing. "Atomic grouping" -(a term taken from Jeffrey Friedl's book) provides the means for specifying -that once a subpattern has matched, it is not to be re-evaluated in this way. -.P -If we use atomic grouping for the previous example, the matcher gives up -immediately on failing to match "foo" the first time. The notation is a kind of -special parenthesis, starting with (?> as in this example: -.sp - (?>\ed+)foo -.sp -This kind of parenthesis "locks up" the part of the pattern it contains once -it has matched, and a failure further into the pattern is prevented from -backtracking into it. Backtracking past it to previous items, however, works as -normal. -.P -An alternative description is that a subpattern of this type matches the string -of characters that an identical standalone pattern would match, if anchored at -the current point in the subject string. -.P -Atomic grouping subpatterns are not capturing subpatterns. Simple cases such as -the above example can be thought of as a maximizing repeat that must swallow -everything it can. So, while both \ed+ and \ed+? are prepared to adjust the -number of digits they match in order to make the rest of the pattern match, -(?>\ed+) can only match an entire sequence of digits. -.P -Atomic groups in general can of course contain arbitrarily complicated -subpatterns, and can be nested. However, when the subpattern for an atomic -group is just a single repeated item, as in the example above, a simpler -notation, called a "possessive quantifier" can be used. This consists of an -additional + character following a quantifier. Using this notation, the -previous example can be rewritten as -.sp - \ed++foo -.sp -Note that a possessive quantifier can be used with an entire group, for -example: -.sp - (abc|xyz){2,3}+ -.sp -Possessive quantifiers are always greedy; the setting of the PCRE_UNGREEDY -option is ignored. They are a convenient notation for the simpler forms of -atomic group. However, there is no difference in the meaning of a possessive -quantifier and the equivalent atomic group, though there may be a performance -difference; possessive quantifiers should be slightly faster. -.P -The possessive quantifier syntax is an extension to the Perl 5.8 syntax. -Jeffrey Friedl originated the idea (and the name) in the first edition of his -book. Mike McCloskey liked it, so implemented it when he built Sun's Java -package, and PCRE copied it from there. It ultimately found its way into Perl -at release 5.10. -.P -PCRE has an optimization that automatically "possessifies" certain simple -pattern constructs. For example, the sequence A+B is treated as A++B because -there is no point in backtracking into a sequence of A's when B must follow. -.P -When a pattern contains an unlimited repeat inside a subpattern that can itself -be repeated an unlimited number of times, the use of an atomic group is the -only way to avoid some failing matches taking a very long time indeed. The -pattern -.sp - (\eD+|<\ed+>)*[!?] -.sp -matches an unlimited number of substrings that either consist of non-digits, or -digits enclosed in <>, followed by either ! or ?. When it matches, it runs -quickly. However, if it is applied to -.sp - aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa -.sp -it takes a long time before reporting failure. This is because the string can -be divided between the internal \eD+ repeat and the external * repeat in a -large number of ways, and all have to be tried. (The example uses [!?] rather -than a single character at the end, because both PCRE and Perl have an -optimization that allows for fast failure when a single character is used. They -remember the last single character that is required for a match, and fail early -if it is not present in the string.) If the pattern is changed so that it uses -an atomic group, like this: -.sp - ((?>\eD+)|<\ed+>)*[!?] -.sp -sequences of non-digits cannot be broken, and failure happens quickly. -. -. -.\" HTML <a name="backreferences"></a> -.SH "BACK REFERENCES" -.rs -.sp -Outside a character class, a backslash followed by a digit greater than 0 (and -possibly further digits) is a back reference to a capturing subpattern earlier -(that is, to its left) in the pattern, provided there have been that many -previous capturing left parentheses. -.P -However, if the decimal number following the backslash is less than 10, it is -always taken as a back reference, and causes an error only if there are not -that many capturing left parentheses in the entire pattern. In other words, the -parentheses that are referenced need not be to the left of the reference for -numbers less than 10. A "forward back reference" of this type can make sense -when a repetition is involved and the subpattern to the right has participated -in an earlier iteration. -.P -It is not possible to have a numerical "forward back reference" to a subpattern -whose number is 10 or more using this syntax because a sequence such as \e50 is -interpreted as a character defined in octal. See the subsection entitled -"Non-printing characters" -.\" HTML <a href="#digitsafterbackslash"> -.\" </a> -above -.\" -for further details of the handling of digits following a backslash. There is -no such problem when named parentheses are used. A back reference to any -subpattern is possible using named parentheses (see below). -.P -Another way of avoiding the ambiguity inherent in the use of digits following a -backslash is to use the \eg escape sequence. This escape must be followed by an -unsigned number or a negative number, optionally enclosed in braces. These -examples are all identical: -.sp - (ring), \e1 - (ring), \eg1 - (ring), \eg{1} -.sp -An unsigned number specifies an absolute reference without the ambiguity that -is present in the older syntax. It is also useful when literal digits follow -the reference. A negative number is a relative reference. Consider this -example: -.sp - (abc(def)ghi)\eg{-1} -.sp -The sequence \eg{-1} is a reference to the most recently started capturing -subpattern before \eg, that is, is it equivalent to \e2 in this example. -Similarly, \eg{-2} would be equivalent to \e1. The use of relative references -can be helpful in long patterns, and also in patterns that are created by -joining together fragments that contain references within themselves. -.P -A back reference matches whatever actually matched the capturing subpattern in -the current subject string, rather than anything matching the subpattern -itself (see -.\" HTML <a href="#subpatternsassubroutines"> -.\" </a> -"Subpatterns as subroutines" -.\" -below for a way of doing that). So the pattern -.sp - (sens|respons)e and \e1ibility -.sp -matches "sense and sensibility" and "response and responsibility", but not -"sense and responsibility". If caseful matching is in force at the time of the -back reference, the case of letters is relevant. For example, -.sp - ((?i)rah)\es+\e1 -.sp -matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original -capturing subpattern is matched caselessly. -.P -There are several different ways of writing back references to named -subpatterns. The .NET syntax \ek{name} and the Perl syntax \ek<name> or -\ek'name' are supported, as is the Python syntax (?P=name). Perl 5.10's unified -back reference syntax, in which \eg can be used for both numeric and named -references, is also supported. We could rewrite the above example in any of -the following ways: -.sp - (?<p1>(?i)rah)\es+\ek<p1> - (?'p1'(?i)rah)\es+\ek{p1} - (?P<p1>(?i)rah)\es+(?P=p1) - (?<p1>(?i)rah)\es+\eg{p1} -.sp -A subpattern that is referenced by name may appear in the pattern before or -after the reference. -.P -There may be more than one back reference to the same subpattern. If a -subpattern has not actually been used in a particular match, any back -references to it always fail by default. For example, the pattern -.sp - (a|(bc))\e2 -.sp -always fails if it starts to match "a" rather than "bc". However, if the -PCRE_JAVASCRIPT_COMPAT option is set at compile time, a back reference to an -unset value matches an empty string. -.P -Because there may be many capturing parentheses in a pattern, all digits -following a backslash are taken as part of a potential back reference number. -If the pattern continues with a digit character, some delimiter must be used to -terminate the back reference. If the PCRE_EXTENDED option is set, this can be -white space. Otherwise, the \eg{ syntax or an empty comment (see -.\" HTML <a href="#comments"> -.\" </a> -"Comments" -.\" -below) can be used. -. -.SS "Recursive back references" -.rs -.sp -A back reference that occurs inside the parentheses to which it refers fails -when the subpattern is first used, so, for example, (a\e1) never matches. -However, such references can be useful inside repeated subpatterns. For -example, the pattern -.sp - (a|b\e1)+ -.sp -matches any number of "a"s and also "aba", "ababbaa" etc. At each iteration of -the subpattern, the back reference matches the character string corresponding -to the previous iteration. In order for this to work, the pattern must be such -that the first iteration does not need to match the back reference. This can be -done using alternation, as in the example above, or by a quantifier with a -minimum of zero. -.P -Back references of this type cause the group that they reference to be treated -as an -.\" HTML <a href="#atomicgroup"> -.\" </a> -atomic group. -.\" -Once the whole group has been matched, a subsequent matching failure cannot -cause backtracking into the middle of the group. -. -. -.\" HTML <a name="bigassertions"></a> -.SH ASSERTIONS -.rs -.sp -An assertion is a test on the characters following or preceding the current -matching point that does not actually consume any characters. The simple -assertions coded as \eb, \eB, \eA, \eG, \eZ, \ez, ^ and $ are described -.\" HTML <a href="#smallassertions"> -.\" </a> -above. -.\" -.P -More complicated assertions are coded as subpatterns. There are two kinds: -those that look ahead of the current position in the subject string, and those -that look behind it. An assertion subpattern is matched in the normal way, -except that it does not cause the current matching position to be changed. -.P -Assertion subpatterns are not capturing subpatterns. If such an assertion -contains capturing subpatterns within it, these are counted for the purposes of -numbering the capturing subpatterns in the whole pattern. However, substring -capturing is carried out only for positive assertions. (Perl sometimes, but not -always, does do capturing in negative assertions.) -.P -For compatibility with Perl, assertion subpatterns may be repeated; though -it makes no sense to assert the same thing several times, the side effect of -capturing parentheses may occasionally be useful. In practice, there only three -cases: -.sp -(1) If the quantifier is {0}, the assertion is never obeyed during matching. -However, it may contain internal capturing parenthesized groups that are called -from elsewhere via the -.\" HTML <a href="#subpatternsassubroutines"> -.\" </a> -subroutine mechanism. -.\" -.sp -(2) If quantifier is {0,n} where n is greater than zero, it is treated as if it -were {0,1}. At run time, the rest of the pattern match is tried with and -without the assertion, the order depending on the greediness of the quantifier. -.sp -(3) If the minimum repetition is greater than zero, the quantifier is ignored. -The assertion is obeyed just once when encountered during matching. -. -. -.SS "Lookahead assertions" -.rs -.sp -Lookahead assertions start with (?= for positive assertions and (?! for -negative assertions. For example, -.sp - \ew+(?=;) -.sp -matches a word followed by a semicolon, but does not include the semicolon in -the match, and -.sp - foo(?!bar) -.sp -matches any occurrence of "foo" that is not followed by "bar". Note that the -apparently similar pattern -.sp - (?!foo)bar -.sp -does not find an occurrence of "bar" that is preceded by something other than -"foo"; it finds any occurrence of "bar" whatsoever, because the assertion -(?!foo) is always true when the next three characters are "bar". A -lookbehind assertion is needed to achieve the other effect. -.P -If you want to force a matching failure at some point in a pattern, the most -convenient way to do it is with (?!) because an empty string always matches, so -an assertion that requires there not to be an empty string must always fail. -The backtracking control verb (*FAIL) or (*F) is a synonym for (?!). -. -. -.\" HTML <a name="lookbehind"></a> -.SS "Lookbehind assertions" -.rs -.sp -Lookbehind assertions start with (?<= for positive assertions and (?<! for -negative assertions. For example, -.sp - (?<!foo)bar -.sp -does find an occurrence of "bar" that is not preceded by "foo". The contents of -a lookbehind assertion are restricted such that all the strings it matches must -have a fixed length. However, if there are several top-level alternatives, they -do not all have to have the same fixed length. Thus -.sp - (?<=bullock|donkey) -.sp -is permitted, but -.sp - (?<!dogs?|cats?) -.sp -causes an error at compile time. Branches that match different length strings -are permitted only at the top level of a lookbehind assertion. This is an -extension compared with Perl, which requires all branches to match the same -length of string. An assertion such as -.sp - (?<=ab(c|de)) -.sp -is not permitted, because its single top-level branch can match two different -lengths, but it is acceptable to PCRE if rewritten to use two top-level -branches: -.sp - (?<=abc|abde) -.sp -In some cases, the escape sequence \eK -.\" HTML <a href="#resetmatchstart"> -.\" </a> -(see above) -.\" -can be used instead of a lookbehind assertion to get round the fixed-length -restriction. -.P -The implementation of lookbehind assertions is, for each alternative, to -temporarily move the current position back by the fixed length and then try to -match. If there are insufficient characters before the current position, the -assertion fails. -.P -In a UTF mode, PCRE does not allow the \eC escape (which matches a single data -unit even in a UTF mode) to appear in lookbehind assertions, because it makes -it impossible to calculate the length of the lookbehind. The \eX and \eR -escapes, which can match different numbers of data units, are also not -permitted. -.P -.\" HTML <a href="#subpatternsassubroutines"> -.\" </a> -"Subroutine" -.\" -calls (see below) such as (?2) or (?&X) are permitted in lookbehinds, as long -as the subpattern matches a fixed-length string. -.\" HTML <a href="#recursion"> -.\" </a> -Recursion, -.\" -however, is not supported. -.P -Possessive quantifiers can be used in conjunction with lookbehind assertions to -specify efficient matching of fixed-length strings at the end of subject -strings. Consider a simple pattern such as -.sp - abcd$ -.sp -when applied to a long string that does not match. Because matching proceeds -from left to right, PCRE will look for each "a" in the subject and then see if -what follows matches the rest of the pattern. If the pattern is specified as -.sp - ^.*abcd$ -.sp -the initial .* matches the entire string at first, but when this fails (because -there is no following "a"), it backtracks to match all but the last character, -then all but the last two characters, and so on. Once again the search for "a" -covers the entire string, from right to left, so we are no better off. However, -if the pattern is written as -.sp - ^.*+(?<=abcd) -.sp -there can be no backtracking for the .*+ item; it can match only the entire -string. The subsequent lookbehind assertion does a single test on the last four -characters. If it fails, the match fails immediately. For long strings, this -approach makes a significant difference to the processing time. -. -. -.SS "Using multiple assertions" -.rs -.sp -Several assertions (of any sort) may occur in succession. For example, -.sp - (?<=\ed{3})(?<!999)foo -.sp -matches "foo" preceded by three digits that are not "999". Notice that each of -the assertions is applied independently at the same point in the subject -string. First there is a check that the previous three characters are all -digits, and then there is a check that the same three characters are not "999". -This pattern does \fInot\fP match "foo" preceded by six characters, the first -of which are digits and the last three of which are not "999". For example, it -doesn't match "123abcfoo". A pattern to do that is -.sp - (?<=\ed{3}...)(?<!999)foo -.sp -This time the first assertion looks at the preceding six characters, checking -that the first three are digits, and then the second assertion checks that the -preceding three characters are not "999". -.P -Assertions can be nested in any combination. For example, -.sp - (?<=(?<!foo)bar)baz -.sp -matches an occurrence of "baz" that is preceded by "bar" which in turn is not -preceded by "foo", while -.sp - (?<=\ed{3}(?!999)...)foo -.sp -is another pattern that matches "foo" preceded by three digits and any three -characters that are not "999". -. -. -.\" HTML <a name="conditions"></a> -.SH "CONDITIONAL SUBPATTERNS" -.rs -.sp -It is possible to cause the matching process to obey a subpattern -conditionally or to choose between two alternative subpatterns, depending on -the result of an assertion, or whether a specific capturing subpattern has -already been matched. The two possible forms of conditional subpattern are: -.sp - (?(condition)yes-pattern) - (?(condition)yes-pattern|no-pattern) -.sp -If the condition is satisfied, the yes-pattern is used; otherwise the -no-pattern (if present) is used. If there are more than two alternatives in the -subpattern, a compile-time error occurs. Each of the two alternatives may -itself contain nested subpatterns of any form, including conditional -subpatterns; the restriction to two alternatives applies only at the level of -the condition. This pattern fragment is an example where the alternatives are -complex: -.sp - (?(1) (A|B|C) | (D | (?(2)E|F) | E) ) -.sp -.P -There are four kinds of condition: references to subpatterns, references to -recursion, a pseudo-condition called DEFINE, and assertions. -. -.SS "Checking for a used subpattern by number" -.rs -.sp -If the text between the parentheses consists of a sequence of digits, the -condition is true if a capturing subpattern of that number has previously -matched. If there is more than one capturing subpattern with the same number -(see the earlier -.\" -.\" HTML <a href="#recursion"> -.\" </a> -section about duplicate subpattern numbers), -.\" -the condition is true if any of them have matched. An alternative notation is -to precede the digits with a plus or minus sign. In this case, the subpattern -number is relative rather than absolute. The most recently opened parentheses -can be referenced by (?(-1), the next most recent by (?(-2), and so on. Inside -loops it can also make sense to refer to subsequent groups. The next -parentheses to be opened can be referenced as (?(+1), and so on. (The value -zero in any of these forms is not used; it provokes a compile-time error.) -.P -Consider the following pattern, which contains non-significant white space to -make it more readable (assume the PCRE_EXTENDED option) and to divide it into -three parts for ease of discussion: -.sp - ( \e( )? [^()]+ (?(1) \e) ) -.sp -The first part matches an optional opening parenthesis, and if that -character is present, sets it as the first captured substring. The second part -matches one or more characters that are not parentheses. The third part is a -conditional subpattern that tests whether or not the first set of parentheses -matched. If they did, that is, if subject started with an opening parenthesis, -the condition is true, and so the yes-pattern is executed and a closing -parenthesis is required. Otherwise, since no-pattern is not present, the -subpattern matches nothing. In other words, this pattern matches a sequence of -non-parentheses, optionally enclosed in parentheses. -.P -If you were embedding this pattern in a larger one, you could use a relative -reference: -.sp - ...other stuff... ( \e( )? [^()]+ (?(-1) \e) ) ... -.sp -This makes the fragment independent of the parentheses in the larger pattern. -. -.SS "Checking for a used subpattern by name" -.rs -.sp -Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a used -subpattern by name. For compatibility with earlier versions of PCRE, which had -this facility before Perl, the syntax (?(name)...) is also recognized. -.P -Rewriting the above example to use a named subpattern gives this: -.sp - (?<OPEN> \e( )? [^()]+ (?(<OPEN>) \e) ) -.sp -If the name used in a condition of this kind is a duplicate, the test is -applied to all subpatterns of the same name, and is true if any one of them has -matched. -. -.SS "Checking for pattern recursion" -.rs -.sp -If the condition is the string (R), and there is no subpattern with the name R, -the condition is true if a recursive call to the whole pattern or any -subpattern has been made. If digits or a name preceded by ampersand follow the -letter R, for example: -.sp - (?(R3)...) or (?(R&name)...) -.sp -the condition is true if the most recent recursion is into a subpattern whose -number or name is given. This condition does not check the entire recursion -stack. If the name used in a condition of this kind is a duplicate, the test is -applied to all subpatterns of the same name, and is true if any one of them is -the most recent recursion. -.P -At "top level", all these recursion test conditions are false. -.\" HTML <a href="#recursion"> -.\" </a> -The syntax for recursive patterns -.\" -is described below. -. -.\" HTML <a name="subdefine"></a> -.SS "Defining subpatterns for use by reference only" -.rs -.sp -If the condition is the string (DEFINE), and there is no subpattern with the -name DEFINE, the condition is always false. In this case, there may be only one -alternative in the subpattern. It is always skipped if control reaches this -point in the pattern; the idea of DEFINE is that it can be used to define -subroutines that can be referenced from elsewhere. (The use of -.\" HTML <a href="#subpatternsassubroutines"> -.\" </a> -subroutines -.\" -is described below.) For example, a pattern to match an IPv4 address such as -"192.168.23.245" could be written like this (ignore white space and line -breaks): -.sp - (?(DEFINE) (?<byte> 2[0-4]\ed | 25[0-5] | 1\ed\ed | [1-9]?\ed) ) - \eb (?&byte) (\e.(?&byte)){3} \eb -.sp -The first part of the pattern is a DEFINE group inside which a another group -named "byte" is defined. This matches an individual component of an IPv4 -address (a number less than 256). When matching takes place, this part of the -pattern is skipped because DEFINE acts like a false condition. The rest of the -pattern uses references to the named group to match the four dot-separated -components of an IPv4 address, insisting on a word boundary at each end. -. -.SS "Assertion conditions" -.rs -.sp -If the condition is not in any of the above formats, it must be an assertion. -This may be a positive or negative lookahead or lookbehind assertion. Consider -this pattern, again containing non-significant white space, and with the two -alternatives on the second line: -.sp - (?(?=[^a-z]*[a-z]) - \ed{2}-[a-z]{3}-\ed{2} | \ed{2}-\ed{2}-\ed{2} ) -.sp -The condition is a positive lookahead assertion that matches an optional -sequence of non-letters followed by a letter. In other words, it tests for the -presence of at least one letter in the subject. If a letter is found, the -subject is matched against the first alternative; otherwise it is matched -against the second. This pattern matches strings in one of the two forms -dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. -. -. -.\" HTML <a name="comments"></a> -.SH COMMENTS -.rs -.sp -There are two ways of including comments in patterns that are processed by -PCRE. In both cases, the start of the comment must not be in a character class, -nor in the middle of any other sequence of related characters such as (?: or a -subpattern name or number. The characters that make up a comment play no part -in the pattern matching. -.P -The sequence (?# marks the start of a comment that continues up to the next -closing parenthesis. Nested parentheses are not permitted. If the PCRE_EXTENDED -option is set, an unescaped # character also introduces a comment, which in -this case continues to immediately after the next newline character or -character sequence in the pattern. Which characters are interpreted as newlines -is controlled by the options passed to a compiling function or by a special -sequence at the start of the pattern, as described in the section entitled -.\" HTML <a href="#newlines"> -.\" </a> -"Newline conventions" -.\" -above. Note that the end of this type of comment is a literal newline sequence -in the pattern; escape sequences that happen to represent a newline do not -count. For example, consider this pattern when PCRE_EXTENDED is set, and the -default newline convention is in force: -.sp - abc #comment \en still comment -.sp -On encountering the # character, \fBpcre_compile()\fP skips along, looking for -a newline in the pattern. The sequence \en is still literal at this stage, so -it does not terminate the comment. Only an actual character with the code value -0x0a (the default newline) does so. -. -. -.\" HTML <a name="recursion"></a> -.SH "RECURSIVE PATTERNS" -.rs -.sp -Consider the problem of matching a string in parentheses, allowing for -unlimited nested parentheses. Without the use of recursion, the best that can -be done is to use a pattern that matches up to some fixed depth of nesting. It -is not possible to handle an arbitrary nesting depth. -.P -For some time, Perl has provided a facility that allows regular expressions to -recurse (amongst other things). It does this by interpolating Perl code in the -expression at run time, and the code can refer to the expression itself. A Perl -pattern using code interpolation to solve the parentheses problem can be -created like this: -.sp - $re = qr{\e( (?: (?>[^()]+) | (?p{$re}) )* \e)}x; -.sp -The (?p{...}) item interpolates Perl code at run time, and in this case refers -recursively to the pattern in which it appears. -.P -Obviously, PCRE cannot support the interpolation of Perl code. Instead, it -supports special syntax for recursion of the entire pattern, and also for -individual subpattern recursion. After its introduction in PCRE and Python, -this kind of recursion was subsequently introduced into Perl at release 5.10. -.P -A special item that consists of (? followed by a number greater than zero and a -closing parenthesis is a recursive subroutine call of the subpattern of the -given number, provided that it occurs inside that subpattern. (If not, it is a -.\" HTML <a href="#subpatternsassubroutines"> -.\" </a> -non-recursive subroutine -.\" -call, which is described in the next section.) The special item (?R) or (?0) is -a recursive call of the entire regular expression. -.P -This PCRE pattern solves the nested parentheses problem (assume the -PCRE_EXTENDED option is set so that white space is ignored): -.sp - \e( ( [^()]++ | (?R) )* \e) -.sp -First it matches an opening parenthesis. Then it matches any number of -substrings which can either be a sequence of non-parentheses, or a recursive -match of the pattern itself (that is, a correctly parenthesized substring). -Finally there is a closing parenthesis. Note the use of a possessive quantifier -to avoid backtracking into sequences of non-parentheses. -.P -If this were part of a larger pattern, you would not want to recurse the entire -pattern, so instead you could use this: -.sp - ( \e( ( [^()]++ | (?1) )* \e) ) -.sp -We have put the pattern into parentheses, and caused the recursion to refer to -them instead of the whole pattern. -.P -In a larger pattern, keeping track of parenthesis numbers can be tricky. This -is made easier by the use of relative references. Instead of (?1) in the -pattern above you can write (?-2) to refer to the second most recently opened -parentheses preceding the recursion. In other words, a negative number counts -capturing parentheses leftwards from the point at which it is encountered. -.P -It is also possible to refer to subsequently opened parentheses, by writing -references such as (?+2). However, these cannot be recursive because the -reference is not inside the parentheses that are referenced. They are always -.\" HTML <a href="#subpatternsassubroutines"> -.\" </a> -non-recursive subroutine -.\" -calls, as described in the next section. -.P -An alternative approach is to use named parentheses instead. The Perl syntax -for this is (?&name); PCRE's earlier syntax (?P>name) is also supported. We -could rewrite the above example as follows: -.sp - (?<pn> \e( ( [^()]++ | (?&pn) )* \e) ) -.sp -If there is more than one subpattern with the same name, the earliest one is -used. -.P -This particular example pattern that we have been looking at contains nested -unlimited repeats, and so the use of a possessive quantifier for matching -strings of non-parentheses is important when applying the pattern to strings -that do not match. For example, when this pattern is applied to -.sp - (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() -.sp -it yields "no match" quickly. However, if a possessive quantifier is not used, -the match runs for a very long time indeed because there are so many different -ways the + and * repeats can carve up the subject, and all have to be tested -before failure can be reported. -.P -At the end of a match, the values of capturing parentheses are those from -the outermost level. If you want to obtain intermediate values, a callout -function can be used (see below and the -.\" HREF -\fBpcrecallout\fP -.\" -documentation). If the pattern above is matched against -.sp - (ab(cd)ef) -.sp -the value for the inner capturing parentheses (numbered 2) is "ef", which is -the last value taken on at the top level. If a capturing subpattern is not -matched at the top level, its final captured value is unset, even if it was -(temporarily) set at a deeper level during the matching process. -.P -If there are more than 15 capturing parentheses in a pattern, PCRE has to -obtain extra memory to store data during a recursion, which it does by using -\fBpcre_malloc\fP, freeing it via \fBpcre_free\fP afterwards. If no memory can -be obtained, the match fails with the PCRE_ERROR_NOMEMORY error. -.P -Do not confuse the (?R) item with the condition (R), which tests for recursion. -Consider this pattern, which matches text in angle brackets, allowing for -arbitrary nesting. Only digits are allowed in nested brackets (that is, when -recursing), whereas any characters are permitted at the outer level. -.sp - < (?: (?(R) \ed++ | [^<>]*+) | (?R)) * > -.sp -In this pattern, (?(R) is the start of a conditional subpattern, with two -different alternatives for the recursive and non-recursive cases. The (?R) item -is the actual recursive call. -. -. -.\" HTML <a name="recursiondifference"></a> -.SS "Differences in recursion processing between PCRE and Perl" -.rs -.sp -Recursion processing in PCRE differs from Perl in two important ways. In PCRE -(like Python, but unlike Perl), a recursive subpattern call is always treated -as an atomic group. That is, once it has matched some of the subject string, it -is never re-entered, even if it contains untried alternatives and there is a -subsequent matching failure. This can be illustrated by the following pattern, -which purports to match a palindromic string that contains an odd number of -characters (for example, "a", "aba", "abcba", "abcdcba"): -.sp - ^(.|(.)(?1)\e2)$ -.sp -The idea is that it either matches a single character, or two identical -characters surrounding a sub-palindrome. In Perl, this pattern works; in PCRE -it does not if the pattern is longer than three characters. Consider the -subject string "abcba": -.P -At the top level, the first character is matched, but as it is not at the end -of the string, the first alternative fails; the second alternative is taken -and the recursion kicks in. The recursive call to subpattern 1 successfully -matches the next character ("b"). (Note that the beginning and end of line -tests are not part of the recursion). -.P -Back at the top level, the next character ("c") is compared with what -subpattern 2 matched, which was "a". This fails. Because the recursion is -treated as an atomic group, there are now no backtracking points, and so the -entire match fails. (Perl is able, at this point, to re-enter the recursion and -try the second alternative.) However, if the pattern is written with the -alternatives in the other order, things are different: -.sp - ^((.)(?1)\e2|.)$ -.sp -This time, the recursing alternative is tried first, and continues to recurse -until it runs out of characters, at which point the recursion fails. But this -time we do have another alternative to try at the higher level. That is the big -difference: in the previous case the remaining alternative is at a deeper -recursion level, which PCRE cannot use. -.P -To change the pattern so that it matches all palindromic strings, not just -those with an odd number of characters, it is tempting to change the pattern to -this: -.sp - ^((.)(?1)\e2|.?)$ -.sp -Again, this works in Perl, but not in PCRE, and for the same reason. When a -deeper recursion has matched a single character, it cannot be entered again in -order to match an empty string. The solution is to separate the two cases, and -write out the odd and even cases as alternatives at the higher level: -.sp - ^(?:((.)(?1)\e2|)|((.)(?3)\e4|.)) -.sp -If you want to match typical palindromic phrases, the pattern has to ignore all -non-word characters, which can be done like this: -.sp - ^\eW*+(?:((.)\eW*+(?1)\eW*+\e2|)|((.)\eW*+(?3)\eW*+\e4|\eW*+.\eW*+))\eW*+$ -.sp -If run with the PCRE_CASELESS option, this pattern matches phrases such as "A -man, a plan, a canal: Panama!" and it works well in both PCRE and Perl. Note -the use of the possessive quantifier *+ to avoid backtracking into sequences of -non-word characters. Without this, PCRE takes a great deal longer (ten times or -more) to match typical phrases, and Perl takes so long that you think it has -gone into a loop. -.P -\fBWARNING\fP: The palindrome-matching patterns above work only if the subject -string does not start with a palindrome that is shorter than the entire string. -For example, although "abcba" is correctly matched, if the subject is "ababa", -PCRE finds the palindrome "aba" at the start, then fails at top level because -the end of the string does not follow. Once again, it cannot jump back into the -recursion to try other alternatives, so the entire match fails. -.P -The second way in which PCRE and Perl differ in their recursion processing is -in the handling of captured values. In Perl, when a subpattern is called -recursively or as a subpattern (see the next section), it has no access to any -values that were captured outside the recursion, whereas in PCRE these values -can be referenced. Consider this pattern: -.sp - ^(.)(\e1|a(?2)) -.sp -In PCRE, this pattern matches "bab". The first capturing parentheses match "b", -then in the second group, when the back reference \e1 fails to match "b", the -second alternative matches "a" and then recurses. In the recursion, \e1 does -now match "b" and so the whole match succeeds. In Perl, the pattern fails to -match because inside the recursive call \e1 cannot access the externally set -value. -. -. -.\" HTML <a name="subpatternsassubroutines"></a> -.SH "SUBPATTERNS AS SUBROUTINES" -.rs -.sp -If the syntax for a recursive subpattern call (either by number or by -name) is used outside the parentheses to which it refers, it operates like a -subroutine in a programming language. The called subpattern may be defined -before or after the reference. A numbered reference can be absolute or -relative, as in these examples: -.sp - (...(absolute)...)...(?2)... - (...(relative)...)...(?-1)... - (...(?+1)...(relative)... -.sp -An earlier example pointed out that the pattern -.sp - (sens|respons)e and \e1ibility -.sp -matches "sense and sensibility" and "response and responsibility", but not -"sense and responsibility". If instead the pattern -.sp - (sens|respons)e and (?1)ibility -.sp -is used, it does match "sense and responsibility" as well as the other two -strings. Another example is given in the discussion of DEFINE above. -.P -All subroutine calls, whether recursive or not, are always treated as atomic -groups. That is, once a subroutine has matched some of the subject string, it -is never re-entered, even if it contains untried alternatives and there is a -subsequent matching failure. Any capturing parentheses that are set during the -subroutine call revert to their previous values afterwards. -.P -Processing options such as case-independence are fixed when a subpattern is -defined, so if it is used as a subroutine, such options cannot be changed for -different calls. For example, consider this pattern: -.sp - (abc)(?i:(?-1)) -.sp -It matches "abcabc". It does not match "abcABC" because the change of -processing option does not affect the called subpattern. -. -. -.\" HTML <a name="onigurumasubroutines"></a> -.SH "ONIGURUMA SUBROUTINE SYNTAX" -.rs -.sp -For compatibility with Oniguruma, the non-Perl syntax \eg followed by a name or -a number enclosed either in angle brackets or single quotes, is an alternative -syntax for referencing a subpattern as a subroutine, possibly recursively. Here -are two of the examples used above, rewritten using this syntax: -.sp - (?<pn> \e( ( (?>[^()]+) | \eg<pn> )* \e) ) - (sens|respons)e and \eg'1'ibility -.sp -PCRE supports an extension to Oniguruma: if a number is preceded by a -plus or a minus sign it is taken as a relative reference. For example: -.sp - (abc)(?i:\eg<-1>) -.sp -Note that \eg{...} (Perl syntax) and \eg<...> (Oniguruma syntax) are \fInot\fP -synonymous. The former is a back reference; the latter is a subroutine call. -. -. -.SH CALLOUTS -.rs -.sp -Perl has a feature whereby using the sequence (?{...}) causes arbitrary Perl -code to be obeyed in the middle of matching a regular expression. This makes it -possible, amongst other things, to extract different substrings that match the -same pair of parentheses when there is a repetition. -.P -PCRE provides a similar feature, but of course it cannot obey arbitrary Perl -code. The feature is called "callout". The caller of PCRE provides an external -function by putting its entry point in the global variable \fIpcre_callout\fP -(8-bit library) or \fIpcre[16|32]_callout\fP (16-bit or 32-bit library). -By default, this variable contains NULL, which disables all calling out. -.P -Within a regular expression, (?C) indicates the points at which the external -function is to be called. If you want to identify different callout points, you -can put a number less than 256 after the letter C. The default value is zero. -For example, this pattern has two callout points: -.sp - (?C1)abc(?C2)def -.sp -If the PCRE_AUTO_CALLOUT flag is passed to a compiling function, callouts are -automatically installed before each item in the pattern. They are all numbered -255. If there is a conditional group in the pattern whose condition is an -assertion, an additional callout is inserted just before the condition. An -explicit callout may also be set at this position, as in this example: -.sp - (?(?C9)(?=a)abc|def) -.sp -Note that this applies only to assertion conditions, not to other types of -condition. -.P -During matching, when PCRE reaches a callout point, the external function is -called. It is provided with the number of the callout, the position in the -pattern, and, optionally, one item of data originally supplied by the caller of -the matching function. The callout function may cause matching to proceed, to -backtrack, or to fail altogether. -.P -By default, PCRE implements a number of optimizations at compile time and -matching time, and one side-effect is that sometimes callouts are skipped. If -you need all possible callouts to happen, you need to set options that disable -the relevant optimizations. More details, and a complete description of the -interface to the callout function, are given in the -.\" HREF -\fBpcrecallout\fP -.\" -documentation. -. -. -.\" HTML <a name="backtrackcontrol"></a> -.SH "BACKTRACKING CONTROL" -.rs -.sp -Perl 5.10 introduced a number of "Special Backtracking Control Verbs", which -are still described in the Perl documentation as "experimental and subject to -change or removal in a future version of Perl". It goes on to say: "Their usage -in production code should be noted to avoid problems during upgrades." The same -remarks apply to the PCRE features described in this section. -.P -The new verbs make use of what was previously invalid syntax: an opening -parenthesis followed by an asterisk. They are generally of the form -(*VERB) or (*VERB:NAME). Some may take either form, possibly behaving -differently depending on whether or not a name is present. A name is any -sequence of characters that does not include a closing parenthesis. The maximum -length of name is 255 in the 8-bit library and 65535 in the 16-bit and 32-bit -libraries. If the name is empty, that is, if the closing parenthesis -immediately follows the colon, the effect is as if the colon were not there. -Any number of these verbs may occur in a pattern. -.P -Since these verbs are specifically related to backtracking, most of them can be -used only when the pattern is to be matched using one of the traditional -matching functions, because these use a backtracking algorithm. With the -exception of (*FAIL), which behaves like a failing negative assertion, the -backtracking control verbs cause an error if encountered by a DFA matching -function. -.P -The behaviour of these verbs in -.\" HTML <a href="#btrepeat"> -.\" </a> -repeated groups, -.\" -.\" HTML <a href="#btassert"> -.\" </a> -assertions, -.\" -and in -.\" HTML <a href="#btsub"> -.\" </a> -subpatterns called as subroutines -.\" -(whether or not recursively) is documented below. -. -. -.\" HTML <a name="nooptimize"></a> -.SS "Optimizations that affect backtracking verbs" -.rs -.sp -PCRE contains some optimizations that are used to speed up matching by running -some checks at the start of each match attempt. For example, it may know the -minimum length of matching subject, or that a particular character must be -present. When one of these optimizations bypasses the running of a match, any -included backtracking verbs will not, of course, be processed. You can suppress -the start-of-match optimizations by setting the PCRE_NO_START_OPTIMIZE option -when calling \fBpcre_compile()\fP or \fBpcre_exec()\fP, or by starting the -pattern with (*NO_START_OPT). There is more discussion of this option in the -section entitled -.\" HTML <a href="pcreapi.html#execoptions"> -.\" </a> -"Option bits for \fBpcre_exec()\fP" -.\" -in the -.\" HREF -\fBpcreapi\fP -.\" -documentation. -.P -Experiments with Perl suggest that it too has similar optimizations, sometimes -leading to anomalous results. -. -. -.SS "Verbs that act immediately" -.rs -.sp -The following verbs act as soon as they are encountered. They may not be -followed by a name. -.sp - (*ACCEPT) -.sp -This verb causes the match to end successfully, skipping the remainder of the -pattern. However, when it is inside a subpattern that is called as a -subroutine, only that subpattern is ended successfully. Matching then continues -at the outer level. If (*ACCEPT) in triggered in a positive assertion, the -assertion succeeds; in a negative assertion, the assertion fails. -.P -If (*ACCEPT) is inside capturing parentheses, the data so far is captured. For -example: -.sp - A((?:A|B(*ACCEPT)|C)D) -.sp -This matches "AB", "AAD", or "ACD"; when it matches "AB", "B" is captured by -the outer parentheses. -.sp - (*FAIL) or (*F) -.sp -This verb causes a matching failure, forcing backtracking to occur. It is -equivalent to (?!) but easier to read. The Perl documentation notes that it is -probably useful only when combined with (?{}) or (??{}). Those are, of course, -Perl features that are not present in PCRE. The nearest equivalent is the -callout feature, as for example in this pattern: -.sp - a+(?C)(*FAIL) -.sp -A match with the string "aaaa" always fails, but the callout is taken before -each backtrack happens (in this example, 10 times). -. -. -.SS "Recording which path was taken" -.rs -.sp -There is one verb whose main purpose is to track how a match was arrived at, -though it also has a secondary use in conjunction with advancing the match -starting point (see (*SKIP) below). -.sp - (*MARK:NAME) or (*:NAME) -.sp -A name is always required with this verb. There may be as many instances of -(*MARK) as you like in a pattern, and their names do not have to be unique. -.P -When a match succeeds, the name of the last-encountered (*MARK:NAME), -(*PRUNE:NAME), or (*THEN:NAME) on the matching path is passed back to the -caller as described in the section entitled -.\" HTML <a href="pcreapi.html#extradata"> -.\" </a> -"Extra data for \fBpcre_exec()\fP" -.\" -in the -.\" HREF -\fBpcreapi\fP -.\" -documentation. Here is an example of \fBpcretest\fP output, where the /K -modifier requests the retrieval and outputting of (*MARK) data: -.sp - re> /X(*MARK:A)Y|X(*MARK:B)Z/K - data> XY - 0: XY - MK: A - XZ - 0: XZ - MK: B -.sp -The (*MARK) name is tagged with "MK:" in this output, and in this example it -indicates which of the two alternatives matched. This is a more efficient way -of obtaining this information than putting each alternative in its own -capturing parentheses. -.P -If a verb with a name is encountered in a positive assertion that is true, the -name is recorded and passed back if it is the last-encountered. This does not -happen for negative assertions or failing positive assertions. -.P -After a partial match or a failed match, the last encountered name in the -entire match process is returned. For example: -.sp - re> /X(*MARK:A)Y|X(*MARK:B)Z/K - data> XP - No match, mark = B -.sp -Note that in this unanchored example the mark is retained from the match -attempt that started at the letter "X" in the subject. Subsequent match -attempts starting at "P" and then with an empty string do not get as far as the -(*MARK) item, but nevertheless do not reset it. -.P -If you are interested in (*MARK) values after failed matches, you should -probably set the PCRE_NO_START_OPTIMIZE option -.\" HTML <a href="#nooptimize"> -.\" </a> -(see above) -.\" -to ensure that the match is always attempted. -. -. -.SS "Verbs that act after backtracking" -.rs -.sp -The following verbs do nothing when they are encountered. Matching continues -with what follows, but if there is no subsequent match, causing a backtrack to -the verb, a failure is forced. That is, backtracking cannot pass to the left of -the verb. However, when one of these verbs appears inside an atomic group or an -assertion that is true, its effect is confined to that group, because once the -group has been matched, there is never any backtracking into it. In this -situation, backtracking can "jump back" to the left of the entire atomic group -or assertion. (Remember also, as stated above, that this localization also -applies in subroutine calls.) -.P -These verbs differ in exactly what kind of failure occurs when backtracking -reaches them. The behaviour described below is what happens when the verb is -not in a subroutine or an assertion. Subsequent sections cover these special -cases. -.sp - (*COMMIT) -.sp -This verb, which may not be followed by a name, causes the whole match to fail -outright if there is a later matching failure that causes backtracking to reach -it. Even if the pattern is unanchored, no further attempts to find a match by -advancing the starting point take place. If (*COMMIT) is the only backtracking -verb that is encountered, once it has been passed \fBpcre_exec()\fP is -committed to finding a match at the current starting point, or not at all. For -example: -.sp - a+(*COMMIT)b -.sp -This matches "xxaab" but not "aacaab". It can be thought of as a kind of -dynamic anchor, or "I've started, so I must finish." The name of the most -recently passed (*MARK) in the path is passed back when (*COMMIT) forces a -match failure. -.P -If there is more than one backtracking verb in a pattern, a different one that -follows (*COMMIT) may be triggered first, so merely passing (*COMMIT) during a -match does not always guarantee that a match must be at this starting point. -.P -Note that (*COMMIT) at the start of a pattern is not the same as an anchor, -unless PCRE's start-of-match optimizations are turned off, as shown in this -output from \fBpcretest\fP: -.sp - re> /(*COMMIT)abc/ - data> xyzabc - 0: abc - data> xyzabc\eY - No match -.sp -For this pattern, PCRE knows that any match must start with "a", so the -optimization skips along the subject to "a" before applying the pattern to the -first set of data. The match attempt then succeeds. In the second set of data, -the escape sequence \eY is interpreted by the \fBpcretest\fP program. It causes -the PCRE_NO_START_OPTIMIZE option to be set when \fBpcre_exec()\fP is called. -This disables the optimization that skips along to the first character. The -pattern is now applied starting at "x", and so the (*COMMIT) causes the match -to fail without trying any other starting points. -.sp - (*PRUNE) or (*PRUNE:NAME) -.sp -This verb causes the match to fail at the current starting position in the -subject if there is a later matching failure that causes backtracking to reach -it. If the pattern is unanchored, the normal "bumpalong" advance to the next -starting character then happens. Backtracking can occur as usual to the left of -(*PRUNE), before it is reached, or when matching to the right of (*PRUNE), but -if there is no match to the right, backtracking cannot cross (*PRUNE). In -simple cases, the use of (*PRUNE) is just an alternative to an atomic group or -possessive quantifier, but there are some uses of (*PRUNE) that cannot be -expressed in any other way. In an anchored pattern (*PRUNE) has the same effect -as (*COMMIT). -.P -The behaviour of (*PRUNE:NAME) is the not the same as (*MARK:NAME)(*PRUNE). -It is like (*MARK:NAME) in that the name is remembered for passing back to the -caller. However, (*SKIP:NAME) searches only for names set with (*MARK). -.sp - (*SKIP) -.sp -This verb, when given without a name, is like (*PRUNE), except that if the -pattern is unanchored, the "bumpalong" advance is not to the next character, -but to the position in the subject where (*SKIP) was encountered. (*SKIP) -signifies that whatever text was matched leading up to it cannot be part of a -successful match. Consider: -.sp - a+(*SKIP)b -.sp -If the subject is "aaaac...", after the first match attempt fails (starting at -the first character in the string), the starting point skips on to start the -next attempt at "c". Note that a possessive quantifer does not have the same -effect as this example; although it would suppress backtracking during the -first match attempt, the second attempt would start at the second character -instead of skipping on to "c". -.sp - (*SKIP:NAME) -.sp -When (*SKIP) has an associated name, its behaviour is modified. When it is -triggered, the previous path through the pattern is searched for the most -recent (*MARK) that has the same name. If one is found, the "bumpalong" advance -is to the subject position that corresponds to that (*MARK) instead of to where -(*SKIP) was encountered. If no (*MARK) with a matching name is found, the -(*SKIP) is ignored. -.P -Note that (*SKIP:NAME) searches only for names set by (*MARK:NAME). It ignores -names that are set by (*PRUNE:NAME) or (*THEN:NAME). -.sp - (*THEN) or (*THEN:NAME) -.sp -This verb causes a skip to the next innermost alternative when backtracking -reaches it. That is, it cancels any further backtracking within the current -alternative. Its name comes from the observation that it can be used for a -pattern-based if-then-else block: -.sp - ( COND1 (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ ) ... -.sp -If the COND1 pattern matches, FOO is tried (and possibly further items after -the end of the group if FOO succeeds); on failure, the matcher skips to the -second alternative and tries COND2, without backtracking into COND1. If that -succeeds and BAR fails, COND3 is tried. If subsequently BAZ fails, there are no -more alternatives, so there is a backtrack to whatever came before the entire -group. If (*THEN) is not inside an alternation, it acts like (*PRUNE). -.P -The behaviour of (*THEN:NAME) is the not the same as (*MARK:NAME)(*THEN). -It is like (*MARK:NAME) in that the name is remembered for passing back to the -caller. However, (*SKIP:NAME) searches only for names set with (*MARK). -.P -A subpattern that does not contain a | character is just a part of the -enclosing alternative; it is not a nested alternation with only one -alternative. The effect of (*THEN) extends beyond such a subpattern to the -enclosing alternative. Consider this pattern, where A, B, etc. are complex -pattern fragments that do not contain any | characters at this level: -.sp - A (B(*THEN)C) | D -.sp -If A and B are matched, but there is a failure in C, matching does not -backtrack into A; instead it moves to the next alternative, that is, D. -However, if the subpattern containing (*THEN) is given an alternative, it -behaves differently: -.sp - A (B(*THEN)C | (*FAIL)) | D -.sp -The effect of (*THEN) is now confined to the inner subpattern. After a failure -in C, matching moves to (*FAIL), which causes the whole subpattern to fail -because there are no more alternatives to try. In this case, matching does now -backtrack into A. -.P -Note that a conditional subpattern is not considered as having two -alternatives, because only one is ever used. In other words, the | character in -a conditional subpattern has a different meaning. Ignoring white space, -consider: -.sp - ^.*? (?(?=a) a | b(*THEN)c ) -.sp -If the subject is "ba", this pattern does not match. Because .*? is ungreedy, -it initially matches zero characters. The condition (?=a) then fails, the -character "b" is matched, but "c" is not. At this point, matching does not -backtrack to .*? as might perhaps be expected from the presence of the | -character. The conditional subpattern is part of the single alternative that -comprises the whole pattern, and so the match fails. (If there was a backtrack -into .*?, allowing it to match "b", the match would succeed.) -.P -The verbs just described provide four different "strengths" of control when -subsequent matching fails. (*THEN) is the weakest, carrying on the match at the -next alternative. (*PRUNE) comes next, failing the match at the current -starting position, but allowing an advance to the next character (for an -unanchored pattern). (*SKIP) is similar, except that the advance may be more -than one character. (*COMMIT) is the strongest, causing the entire match to -fail. -. -. -.SS "More than one backtracking verb" -.rs -.sp -If more than one backtracking verb is present in a pattern, the one that is -backtracked onto first acts. For example, consider this pattern, where A, B, -etc. are complex pattern fragments: -.sp - (A(*COMMIT)B(*THEN)C|ABD) -.sp -If A matches but B fails, the backtrack to (*COMMIT) causes the entire match to -fail. However, if A and B match, but C fails, the backtrack to (*THEN) causes -the next alternative (ABD) to be tried. This behaviour is consistent, but is -not always the same as Perl's. It means that if two or more backtracking verbs -appear in succession, all the the last of them has no effect. Consider this -example: -.sp - ...(*COMMIT)(*PRUNE)... -.sp -If there is a matching failure to the right, backtracking onto (*PRUNE) causes -it to be triggered, and its action is taken. There can never be a backtrack -onto (*COMMIT). -. -. -.\" HTML <a name="btrepeat"></a> -.SS "Backtracking verbs in repeated groups" -.rs -.sp -PCRE differs from Perl in its handling of backtracking verbs in repeated -groups. For example, consider: -.sp - /(a(*COMMIT)b)+ac/ -.sp -If the subject is "abac", Perl matches, but PCRE fails because the (*COMMIT) in -the second repeat of the group acts. -. -. -.\" HTML <a name="btassert"></a> -.SS "Backtracking verbs in assertions" -.rs -.sp -(*FAIL) in an assertion has its normal effect: it forces an immediate backtrack. -.P -(*ACCEPT) in a positive assertion causes the assertion to succeed without any -further processing. In a negative assertion, (*ACCEPT) causes the assertion to -fail without any further processing. -.P -The other backtracking verbs are not treated specially if they appear in a -positive assertion. In particular, (*THEN) skips to the next alternative in the -innermost enclosing group that has alternations, whether or not this is within -the assertion. -.P -Negative assertions are, however, different, in order to ensure that changing a -positive assertion into a negative assertion changes its result. Backtracking -into (*COMMIT), (*SKIP), or (*PRUNE) causes a negative assertion to be true, -without considering any further alternative branches in the assertion. -Backtracking into (*THEN) causes it to skip to the next enclosing alternative -within the assertion (the normal behaviour), but if the assertion does not have -such an alternative, (*THEN) behaves like (*PRUNE). -. -. -.\" HTML <a name="btsub"></a> -.SS "Backtracking verbs in subroutines" -.rs -.sp -These behaviours occur whether or not the subpattern is called recursively. -Perl's treatment of subroutines is different in some cases. -.P -(*FAIL) in a subpattern called as a subroutine has its normal effect: it forces -an immediate backtrack. -.P -(*ACCEPT) in a subpattern called as a subroutine causes the subroutine match to -succeed without any further processing. Matching then continues after the -subroutine call. -.P -(*COMMIT), (*SKIP), and (*PRUNE) in a subpattern called as a subroutine cause -the subroutine match to fail. -.P -(*THEN) skips to the next alternative in the innermost enclosing group within -the subpattern that has alternatives. If there is no such group within the -subpattern, (*THEN) causes the subroutine match to fail. -. -. -.SH "SEE ALSO" -.rs -.sp -\fBpcreapi\fP(3), \fBpcrecallout\fP(3), \fBpcrematching\fP(3), -\fBpcresyntax\fP(3), \fBpcre\fP(3), \fBpcre16(3)\fP, \fBpcre32(3)\fP. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 08 January 2014 -Copyright (c) 1997-2014 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcreperform.3 b/plugins/Pcre16/docs/doc/pcreperform.3 deleted file mode 100644 index fb2aa95926..0000000000 --- a/plugins/Pcre16/docs/doc/pcreperform.3 +++ /dev/null @@ -1,177 +0,0 @@ -.TH PCREPERFORM 3 "09 January 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH "PCRE PERFORMANCE" -.rs -.sp -Two aspects of performance are discussed below: memory usage and processing -time. The way you express your pattern as a regular expression can affect both -of them. -. -.SH "COMPILED PATTERN MEMORY USAGE" -.rs -.sp -Patterns are compiled by PCRE into a reasonably efficient interpretive code, so -that most simple patterns do not use much memory. However, there is one case -where the memory usage of a compiled pattern can be unexpectedly large. If a -parenthesized subpattern has a quantifier with a minimum greater than 1 and/or -a limited maximum, the whole subpattern is repeated in the compiled code. For -example, the pattern -.sp - (abc|def){2,4} -.sp -is compiled as if it were -.sp - (abc|def)(abc|def)((abc|def)(abc|def)?)? -.sp -(Technical aside: It is done this way so that backtrack points within each of -the repetitions can be independently maintained.) -.P -For regular expressions whose quantifiers use only small numbers, this is not -usually a problem. However, if the numbers are large, and particularly if such -repetitions are nested, the memory usage can become an embarrassment. For -example, the very simple pattern -.sp - ((ab){1,1000}c){1,3} -.sp -uses 51K bytes when compiled using the 8-bit library. When PCRE is compiled -with its default internal pointer size of two bytes, the size limit on a -compiled pattern is 64K data units, and this is reached with the above pattern -if the outer repetition is increased from 3 to 4. PCRE can be compiled to use -larger internal pointers and thus handle larger compiled patterns, but it is -better to try to rewrite your pattern to use less memory if you can. -.P -One way of reducing the memory usage for such patterns is to make use of PCRE's -.\" HTML <a href="pcrepattern.html#subpatternsassubroutines"> -.\" </a> -"subroutine" -.\" -facility. Re-writing the above pattern as -.sp - ((ab)(?2){0,999}c)(?1){0,2} -.sp -reduces the memory requirements to 18K, and indeed it remains under 20K even -with the outer repetition increased to 100. However, this pattern is not -exactly equivalent, because the "subroutine" calls are treated as -.\" HTML <a href="pcrepattern.html#atomicgroup"> -.\" </a> -atomic groups -.\" -into which there can be no backtracking if there is a subsequent matching -failure. Therefore, PCRE cannot do this kind of rewriting automatically. -Furthermore, there is a noticeable loss of speed when executing the modified -pattern. Nevertheless, if the atomic grouping is not a problem and the loss of -speed is acceptable, this kind of rewriting will allow you to process patterns -that PCRE cannot otherwise handle. -. -. -.SH "STACK USAGE AT RUN TIME" -.rs -.sp -When \fBpcre_exec()\fP or \fBpcre[16|32]_exec()\fP is used for matching, certain -kinds of pattern can cause it to use large amounts of the process stack. In -some environments the default process stack is quite small, and if it runs out -the result is often SIGSEGV. This issue is probably the most frequently raised -problem with PCRE. Rewriting your pattern can often help. The -.\" HREF -\fBpcrestack\fP -.\" -documentation discusses this issue in detail. -. -. -.SH "PROCESSING TIME" -.rs -.sp -Certain items in regular expression patterns are processed more efficiently -than others. It is more efficient to use a character class like [aeiou] than a -set of single-character alternatives such as (a|e|i|o|u). In general, the -simplest construction that provides the required behaviour is usually the most -efficient. Jeffrey Friedl's book contains a lot of useful general discussion -about optimizing regular expressions for efficient performance. This document -contains a few observations about PCRE. -.P -Using Unicode character properties (the \ep, \eP, and \eX escapes) is slow, -because PCRE has to use a multi-stage table lookup whenever it needs a -character's property. If you can find an alternative pattern that does not use -character properties, it will probably be faster. -.P -By default, the escape sequences \eb, \ed, \es, and \ew, and the POSIX -character classes such as [:alpha:] do not use Unicode properties, partly for -backwards compatibility, and partly for performance reasons. However, you can -set PCRE_UCP if you want Unicode character properties to be used. This can -double the matching time for items such as \ed, when matched with -a traditional matching function; the performance loss is less with -a DFA matching function, and in both cases there is not much difference for -\eb. -.P -When a pattern begins with .* not in parentheses, or in parentheses that are -not the subject of a backreference, and the PCRE_DOTALL option is set, the -pattern is implicitly anchored by PCRE, since it can match only at the start of -a subject string. However, if PCRE_DOTALL is not set, PCRE cannot make this -optimization, because the . metacharacter does not then match a newline, and if -the subject string contains newlines, the pattern may match from the character -immediately following one of them instead of from the very start. For example, -the pattern -.sp - .*second -.sp -matches the subject "first\enand second" (where \en stands for a newline -character), with the match starting at the seventh character. In order to do -this, PCRE has to retry the match starting after every newline in the subject. -.P -If you are using such a pattern with subject strings that do not contain -newlines, the best performance is obtained by setting PCRE_DOTALL, or starting -the pattern with ^.* or ^.*? to indicate explicit anchoring. That saves PCRE -from having to scan along the subject looking for a newline to restart at. -.P -Beware of patterns that contain nested indefinite repeats. These can take a -long time to run when applied to a string that does not match. Consider the -pattern fragment -.sp - ^(a+)* -.sp -This can match "aaaa" in 16 different ways, and this number increases very -rapidly as the string gets longer. (The * repeat can match 0, 1, 2, 3, or 4 -times, and for each of those cases other than 0 or 4, the + repeats can match -different numbers of times.) When the remainder of the pattern is such that the -entire match is going to fail, PCRE has in principle to try every possible -variation, and this can take an extremely long time, even for relatively short -strings. -.P -An optimization catches some of the more simple cases such as -.sp - (a+)*b -.sp -where a literal character follows. Before embarking on the standard matching -procedure, PCRE checks that there is a "b" later in the subject string, and if -there is not, it fails the match immediately. However, when there is no -following literal this optimization cannot be used. You can see the difference -by comparing the behaviour of -.sp - (a+)*\ed -.sp -with the pattern above. The former gives a failure almost instantly when -applied to a whole line of "a" characters, whereas the latter takes an -appreciable time with strings longer than about 20 characters. -.P -In many cases, the solution to this kind of performance issue is to use an -atomic group or a possessive quantifier. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 25 August 2012 -Copyright (c) 1997-2012 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcreposix.3 b/plugins/Pcre16/docs/doc/pcreposix.3 deleted file mode 100644 index 77890f36b4..0000000000 --- a/plugins/Pcre16/docs/doc/pcreposix.3 +++ /dev/null @@ -1,267 +0,0 @@ -.TH PCREPOSIX 3 "09 January 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions. -.SH "SYNOPSIS" -.rs -.sp -.B #include <pcreposix.h> -.PP -.nf -.B int regcomp(regex_t *\fIpreg\fP, const char *\fIpattern\fP, -.B " int \fIcflags\fP);" -.sp -.B int regexec(regex_t *\fIpreg\fP, const char *\fIstring\fP, -.B " size_t \fInmatch\fP, regmatch_t \fIpmatch\fP[], int \fIeflags\fP);" -.B " size_t regerror(int \fIerrcode\fP, const regex_t *\fIpreg\fP," -.B " char *\fIerrbuf\fP, size_t \fIerrbuf_size\fP);" -.sp -.B void regfree(regex_t *\fIpreg\fP); -.fi -. -.SH DESCRIPTION -.rs -.sp -This set of functions provides a POSIX-style API for the PCRE regular -expression 8-bit library. See the -.\" HREF -\fBpcreapi\fP -.\" -documentation for a description of PCRE's native API, which contains much -additional functionality. There is no POSIX-style wrapper for PCRE's 16-bit -and 32-bit library. -.P -The functions described here are just wrapper functions that ultimately call -the PCRE native API. Their prototypes are defined in the \fBpcreposix.h\fP -header file, and on Unix systems the library itself is called -\fBpcreposix.a\fP, so can be accessed by adding \fB-lpcreposix\fP to the -command for linking an application that uses them. Because the POSIX functions -call the native ones, it is also necessary to add \fB-lpcre\fP. -.P -I have implemented only those POSIX option bits that can be reasonably mapped -to PCRE native options. In addition, the option REG_EXTENDED is defined with -the value zero. This has no effect, but since programs that are written to the -POSIX interface often use it, this makes it easier to slot in PCRE as a -replacement library. Other POSIX options are not even defined. -.P -There are also some other options that are not defined by POSIX. These have -been added at the request of users who want to make use of certain -PCRE-specific features via the POSIX calling interface. -.P -When PCRE is called via these functions, it is only the API that is POSIX-like -in style. The syntax and semantics of the regular expressions themselves are -still those of Perl, subject to the setting of various PCRE options, as -described below. "POSIX-like in style" means that the API approximates to the -POSIX definition; it is not fully POSIX-compatible, and in multi-byte encoding -domains it is probably even less compatible. -.P -The header for these functions is supplied as \fBpcreposix.h\fP to avoid any -potential clash with other POSIX libraries. It can, of course, be renamed or -aliased as \fBregex.h\fP, which is the "correct" name. It provides two -structure types, \fIregex_t\fP for compiled internal forms, and -\fIregmatch_t\fP for returning captured substrings. It also defines some -constants whose names start with "REG_"; these are used for setting options and -identifying error codes. -. -. -.SH "COMPILING A PATTERN" -.rs -.sp -The function \fBregcomp()\fP is called to compile a pattern into an -internal form. The pattern is a C string terminated by a binary zero, and -is passed in the argument \fIpattern\fP. The \fIpreg\fP argument is a pointer -to a \fBregex_t\fP structure that is used as a base for storing information -about the compiled regular expression. -.P -The argument \fIcflags\fP is either zero, or contains one or more of the bits -defined by the following macros: -.sp - REG_DOTALL -.sp -The PCRE_DOTALL option is set when the regular expression is passed for -compilation to the native function. Note that REG_DOTALL is not part of the -POSIX standard. -.sp - REG_ICASE -.sp -The PCRE_CASELESS option is set when the regular expression is passed for -compilation to the native function. -.sp - REG_NEWLINE -.sp -The PCRE_MULTILINE option is set when the regular expression is passed for -compilation to the native function. Note that this does \fInot\fP mimic the -defined POSIX behaviour for REG_NEWLINE (see the following section). -.sp - REG_NOSUB -.sp -The PCRE_NO_AUTO_CAPTURE option is set when the regular expression is passed -for compilation to the native function. In addition, when a pattern that is -compiled with this flag is passed to \fBregexec()\fP for matching, the -\fInmatch\fP and \fIpmatch\fP arguments are ignored, and no captured strings -are returned. -.sp - REG_UCP -.sp -The PCRE_UCP option is set when the regular expression is passed for -compilation to the native function. This causes PCRE to use Unicode properties -when matchine \ed, \ew, etc., instead of just recognizing ASCII values. Note -that REG_UTF8 is not part of the POSIX standard. -.sp - REG_UNGREEDY -.sp -The PCRE_UNGREEDY option is set when the regular expression is passed for -compilation to the native function. Note that REG_UNGREEDY is not part of the -POSIX standard. -.sp - REG_UTF8 -.sp -The PCRE_UTF8 option is set when the regular expression is passed for -compilation to the native function. This causes the pattern itself and all data -strings used for matching it to be treated as UTF-8 strings. Note that REG_UTF8 -is not part of the POSIX standard. -.P -In the absence of these flags, no options are passed to the native function. -This means the the regex is compiled with PCRE default semantics. In -particular, the way it handles newline characters in the subject string is the -Perl way, not the POSIX way. Note that setting PCRE_MULTILINE has only -\fIsome\fP of the effects specified for REG_NEWLINE. It does not affect the way -newlines are matched by . (they are not) or by a negative class such as [^a] -(they are). -.P -The yield of \fBregcomp()\fP is zero on success, and non-zero otherwise. The -\fIpreg\fP structure is filled in on success, and one member of the structure -is public: \fIre_nsub\fP contains the number of capturing subpatterns in -the regular expression. Various error codes are defined in the header file. -.P -NOTE: If the yield of \fBregcomp()\fP is non-zero, you must not attempt to -use the contents of the \fIpreg\fP structure. If, for example, you pass it to -\fBregexec()\fP, the result is undefined and your program is likely to crash. -. -. -.SH "MATCHING NEWLINE CHARACTERS" -.rs -.sp -This area is not simple, because POSIX and Perl take different views of things. -It is not possible to get PCRE to obey POSIX semantics, but then PCRE was never -intended to be a POSIX engine. The following table lists the different -possibilities for matching newline characters in PCRE: -.sp - Default Change with -.sp - . matches newline no PCRE_DOTALL - newline matches [^a] yes not changeable - $ matches \en at end yes PCRE_DOLLARENDONLY - $ matches \en in middle no PCRE_MULTILINE - ^ matches \en in middle no PCRE_MULTILINE -.sp -This is the equivalent table for POSIX: -.sp - Default Change with -.sp - . matches newline yes REG_NEWLINE - newline matches [^a] yes REG_NEWLINE - $ matches \en at end no REG_NEWLINE - $ matches \en in middle no REG_NEWLINE - ^ matches \en in middle no REG_NEWLINE -.sp -PCRE's behaviour is the same as Perl's, except that there is no equivalent for -PCRE_DOLLAR_ENDONLY in Perl. In both PCRE and Perl, there is no way to stop -newline from matching [^a]. -.P -The default POSIX newline handling can be obtained by setting PCRE_DOTALL and -PCRE_DOLLAR_ENDONLY, but there is no way to make PCRE behave exactly as for the -REG_NEWLINE action. -. -. -.SH "MATCHING A PATTERN" -.rs -.sp -The function \fBregexec()\fP is called to match a compiled pattern \fIpreg\fP -against a given \fIstring\fP, which is by default terminated by a zero byte -(but see REG_STARTEND below), subject to the options in \fIeflags\fP. These can -be: -.sp - REG_NOTBOL -.sp -The PCRE_NOTBOL option is set when calling the underlying PCRE matching -function. -.sp - REG_NOTEMPTY -.sp -The PCRE_NOTEMPTY option is set when calling the underlying PCRE matching -function. Note that REG_NOTEMPTY is not part of the POSIX standard. However, -setting this option can give more POSIX-like behaviour in some situations. -.sp - REG_NOTEOL -.sp -The PCRE_NOTEOL option is set when calling the underlying PCRE matching -function. -.sp - REG_STARTEND -.sp -The string is considered to start at \fIstring\fP + \fIpmatch[0].rm_so\fP and -to have a terminating NUL located at \fIstring\fP + \fIpmatch[0].rm_eo\fP -(there need not actually be a NUL at that location), regardless of the value of -\fInmatch\fP. This is a BSD extension, compatible with but not specified by -IEEE Standard 1003.2 (POSIX.2), and should be used with caution in software -intended to be portable to other systems. Note that a non-zero \fIrm_so\fP does -not imply REG_NOTBOL; REG_STARTEND affects only the location of the string, not -how it is matched. -.P -If the pattern was compiled with the REG_NOSUB flag, no data about any matched -strings is returned. The \fInmatch\fP and \fIpmatch\fP arguments of -\fBregexec()\fP are ignored. -.P -If the value of \fInmatch\fP is zero, or if the value \fIpmatch\fP is NULL, -no data about any matched strings is returned. -.P -Otherwise,the portion of the string that was matched, and also any captured -substrings, are returned via the \fIpmatch\fP argument, which points to an -array of \fInmatch\fP structures of type \fIregmatch_t\fP, containing the -members \fIrm_so\fP and \fIrm_eo\fP. These contain the offset to the first -character of each substring and the offset to the first character after the end -of each substring, respectively. The 0th element of the vector relates to the -entire portion of \fIstring\fP that was matched; subsequent elements relate to -the capturing subpatterns of the regular expression. Unused entries in the -array have both structure members set to -1. -.P -A successful match yields a zero return; various error codes are defined in the -header file, of which REG_NOMATCH is the "expected" failure code. -. -. -.SH "ERROR MESSAGES" -.rs -.sp -The \fBregerror()\fP function maps a non-zero errorcode from either -\fBregcomp()\fP or \fBregexec()\fP to a printable message. If \fIpreg\fP is not -NULL, the error should have arisen from the use of that structure. A message -terminated by a binary zero is placed in \fIerrbuf\fP. The length of the -message, including the zero, is limited to \fIerrbuf_size\fP. The yield of the -function is the size of buffer needed to hold the whole message. -. -. -.SH MEMORY USAGE -.rs -.sp -Compiling a regular expression causes memory to be allocated and associated -with the \fIpreg\fP structure. The function \fBregfree()\fP frees all such -memory, after which \fIpreg\fP may no longer be used as a compiled expression. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 09 January 2012 -Copyright (c) 1997-2012 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcreprecompile.3 b/plugins/Pcre16/docs/doc/pcreprecompile.3 deleted file mode 100644 index 40f257a98c..0000000000 --- a/plugins/Pcre16/docs/doc/pcreprecompile.3 +++ /dev/null @@ -1,155 +0,0 @@ -.TH PCREPRECOMPILE 3 "12 November 2013" "PCRE 8.34" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH "SAVING AND RE-USING PRECOMPILED PCRE PATTERNS" -.rs -.sp -If you are running an application that uses a large number of regular -expression patterns, it may be useful to store them in a precompiled form -instead of having to compile them every time the application is run. -If you are not using any private character tables (see the -.\" HREF -\fBpcre_maketables()\fP -.\" -documentation), this is relatively straightforward. If you are using private -tables, it is a little bit more complicated. However, if you are using the -just-in-time optimization feature, it is not possible to save and reload the -JIT data. -.P -If you save compiled patterns to a file, you can copy them to a different host -and run them there. If the two hosts have different endianness (byte order), -you should run the \fBpcre[16|32]_pattern_to_host_byte_order()\fP function on the -new host before trying to match the pattern. The matching functions return -PCRE_ERROR_BADENDIANNESS if they detect a pattern with the wrong endianness. -.P -Compiling regular expressions with one version of PCRE for use with a different -version is not guaranteed to work and may cause crashes, and saving and -restoring a compiled pattern loses any JIT optimization data. -. -. -.SH "SAVING A COMPILED PATTERN" -.rs -.sp -The value returned by \fBpcre[16|32]_compile()\fP points to a single block of -memory that holds the compiled pattern and associated data. You can find the -length of this block in bytes by calling \fBpcre[16|32]_fullinfo()\fP with an -argument of PCRE_INFO_SIZE. You can then save the data in any appropriate -manner. Here is sample code for the 8-bit library that compiles a pattern and -writes it to a file. It assumes that the variable \fIfd\fP refers to a file -that is open for output: -.sp - int erroroffset, rc, size; - char *error; - pcre *re; -.sp - re = pcre_compile("my pattern", 0, &error, &erroroffset, NULL); - if (re == NULL) { ... handle errors ... } - rc = pcre_fullinfo(re, NULL, PCRE_INFO_SIZE, &size); - if (rc < 0) { ... handle errors ... } - rc = fwrite(re, 1, size, fd); - if (rc != size) { ... handle errors ... } -.sp -In this example, the bytes that comprise the compiled pattern are copied -exactly. Note that this is binary data that may contain any of the 256 possible -byte values. On systems that make a distinction between binary and non-binary -data, be sure that the file is opened for binary output. -.P -If you want to write more than one pattern to a file, you will have to devise a -way of separating them. For binary data, preceding each pattern with its length -is probably the most straightforward approach. Another possibility is to write -out the data in hexadecimal instead of binary, one pattern to a line. -.P -Saving compiled patterns in a file is only one possible way of storing them for -later use. They could equally well be saved in a database, or in the memory of -some daemon process that passes them via sockets to the processes that want -them. -.P -If the pattern has been studied, it is also possible to save the normal study -data in a similar way to the compiled pattern itself. However, if the -PCRE_STUDY_JIT_COMPILE was used, the just-in-time data that is created cannot -be saved because it is too dependent on the current environment. When studying -generates additional information, \fBpcre[16|32]_study()\fP returns a pointer to a -\fBpcre[16|32]_extra\fP data block. Its format is defined in the -.\" HTML <a href="pcreapi.html#extradata"> -.\" </a> -section on matching a pattern -.\" -in the -.\" HREF -\fBpcreapi\fP -.\" -documentation. The \fIstudy_data\fP field points to the binary study data, and -this is what you must save (not the \fBpcre[16|32]_extra\fP block itself). The -length of the study data can be obtained by calling \fBpcre[16|32]_fullinfo()\fP -with an argument of PCRE_INFO_STUDYSIZE. Remember to check that -\fBpcre[16|32]_study()\fP did return a non-NULL value before trying to save the -study data. -. -. -.SH "RE-USING A PRECOMPILED PATTERN" -.rs -.sp -Re-using a precompiled pattern is straightforward. Having reloaded it into main -memory, called \fBpcre[16|32]_pattern_to_host_byte_order()\fP if necessary, you -pass its pointer to \fBpcre[16|32]_exec()\fP or \fBpcre[16|32]_dfa_exec()\fP in -the usual way. -.P -However, if you passed a pointer to custom character tables when the pattern -was compiled (the \fItableptr\fP argument of \fBpcre[16|32]_compile()\fP), you -must now pass a similar pointer to \fBpcre[16|32]_exec()\fP or -\fBpcre[16|32]_dfa_exec()\fP, because the value saved with the compiled pattern -will obviously be nonsense. A field in a \fBpcre[16|32]_extra()\fP block is used -to pass this data, as described in the -.\" HTML <a href="pcreapi.html#extradata"> -.\" </a> -section on matching a pattern -.\" -in the -.\" HREF -\fBpcreapi\fP -.\" -documentation. -.P -\fBWarning:\fP The tables that \fBpcre_exec()\fP and \fBpcre_dfa_exec()\fP use -must be the same as those that were used when the pattern was compiled. If this -is not the case, the behaviour is undefined. -.P -If you did not provide custom character tables when the pattern was compiled, -the pointer in the compiled pattern is NULL, which causes the matching -functions to use PCRE's internal tables. Thus, you do not need to take any -special action at run time in this case. -.P -If you saved study data with the compiled pattern, you need to create your own -\fBpcre[16|32]_extra\fP data block and set the \fIstudy_data\fP field to point -to the reloaded study data. You must also set the PCRE_EXTRA_STUDY_DATA bit in -the \fIflags\fP field to indicate that study data is present. Then pass the -\fBpcre[16|32]_extra\fP block to the matching function in the usual way. If the -pattern was studied for just-in-time optimization, that data cannot be saved, -and so is lost by a save/restore cycle. -. -. -.SH "COMPATIBILITY WITH DIFFERENT PCRE RELEASES" -.rs -.sp -In general, it is safest to recompile all saved patterns when you update to a -new PCRE release, though not all updates actually require this. -. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 12 November 2013 -Copyright (c) 1997-2013 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcresample.3 b/plugins/Pcre16/docs/doc/pcresample.3 deleted file mode 100644 index d7fe7ec546..0000000000 --- a/plugins/Pcre16/docs/doc/pcresample.3 +++ /dev/null @@ -1,99 +0,0 @@ -.TH PCRESAMPLE 3 "10 January 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH "PCRE SAMPLE PROGRAM" -.rs -.sp -A simple, complete demonstration program, to get you started with using PCRE, -is supplied in the file \fIpcredemo.c\fP in the PCRE distribution. A listing of -this program is given in the -.\" HREF -\fBpcredemo\fP -.\" -documentation. If you do not have a copy of the PCRE distribution, you can save -this listing to re-create \fIpcredemo.c\fP. -.P -The demonstration program, which uses the original PCRE 8-bit library, compiles -the regular expression that is its first argument, and matches it against the -subject string in its second argument. No PCRE options are set, and default -character tables are used. If matching succeeds, the program outputs the -portion of the subject that matched, together with the contents of any captured -substrings. -.P -If the -g option is given on the command line, the program then goes on to -check for further matches of the same regular expression in the same subject -string. The logic is a little bit tricky because of the possibility of matching -an empty string. Comments in the code explain what is going on. -.P -If PCRE is installed in the standard include and library directories for your -operating system, you should be able to compile the demonstration program using -this command: -.sp - gcc -o pcredemo pcredemo.c -lpcre -.sp -If PCRE is installed elsewhere, you may need to add additional options to the -command line. For example, on a Unix-like system that has PCRE installed in -\fI/usr/local\fP, you can compile the demonstration program using a command -like this: -.sp -.\" JOINSH - gcc -o pcredemo -I/usr/local/include pcredemo.c \e - -L/usr/local/lib -lpcre -.sp -In a Windows environment, if you want to statically link the program against a -non-dll \fBpcre.a\fP file, you must uncomment the line that defines PCRE_STATIC -before including \fBpcre.h\fP, because otherwise the \fBpcre_malloc()\fP and -\fBpcre_free()\fP exported functions will be declared -\fB__declspec(dllimport)\fP, with unwanted results. -.P -Once you have compiled and linked the demonstration program, you can run simple -tests like this: -.sp - ./pcredemo 'cat|dog' 'the cat sat on the mat' - ./pcredemo -g 'cat|dog' 'the dog sat on the cat' -.sp -Note that there is a much more comprehensive test program, called -.\" HREF -\fBpcretest\fP, -.\" -which supports many more facilities for testing regular expressions and both -PCRE libraries. The -.\" HREF -\fBpcredemo\fP -.\" -program is provided as a simple coding example. -.P -If you try to run -.\" HREF -\fBpcredemo\fP -.\" -when PCRE is not installed in the standard library directory, you may get an -error like this on some operating systems (e.g. Solaris): -.sp - ld.so.1: a.out: fatal: libpcre.so.0: open failed: No such file or directory -.sp -This is caused by the way shared library support works on those systems. You -need to add -.sp - -R/usr/local/lib -.sp -(for example) to the compile command to get round this problem. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 10 January 2012 -Copyright (c) 1997-2012 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcrestack.3 b/plugins/Pcre16/docs/doc/pcrestack.3 deleted file mode 100644 index 798f0bca63..0000000000 --- a/plugins/Pcre16/docs/doc/pcrestack.3 +++ /dev/null @@ -1,215 +0,0 @@ -.TH PCRESTACK 3 "24 June 2012" "PCRE 8.30" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH "PCRE DISCUSSION OF STACK USAGE" -.rs -.sp -When you call \fBpcre[16|32]_exec()\fP, it makes use of an internal function -called \fBmatch()\fP. This calls itself recursively at branch points in the -pattern, in order to remember the state of the match so that it can back up and -try a different alternative if the first one fails. As matching proceeds deeper -and deeper into the tree of possibilities, the recursion depth increases. The -\fBmatch()\fP function is also called in other circumstances, for example, -whenever a parenthesized sub-pattern is entered, and in certain cases of -repetition. -.P -Not all calls of \fBmatch()\fP increase the recursion depth; for an item such -as a* it may be called several times at the same level, after matching -different numbers of a's. Furthermore, in a number of cases where the result of -the recursive call would immediately be passed back as the result of the -current call (a "tail recursion"), the function is just restarted instead. -.P -The above comments apply when \fBpcre[16|32]_exec()\fP is run in its normal -interpretive manner. If the pattern was studied with the -PCRE_STUDY_JIT_COMPILE option, and just-in-time compiling was successful, and -the options passed to \fBpcre[16|32]_exec()\fP were not incompatible, the matching -process uses the JIT-compiled code instead of the \fBmatch()\fP function. In -this case, the memory requirements are handled entirely differently. See the -.\" HREF -\fBpcrejit\fP -.\" -documentation for details. -.P -The \fBpcre[16|32]_dfa_exec()\fP function operates in an entirely different way, -and uses recursion only when there is a regular expression recursion or -subroutine call in the pattern. This includes the processing of assertion and -"once-only" subpatterns, which are handled like subroutine calls. Normally, -these are never very deep, and the limit on the complexity of -\fBpcre[16|32]_dfa_exec()\fP is controlled by the amount of workspace it is given. -However, it is possible to write patterns with runaway infinite recursions; -such patterns will cause \fBpcre[16|32]_dfa_exec()\fP to run out of stack. At -present, there is no protection against this. -.P -The comments that follow do NOT apply to \fBpcre[16|32]_dfa_exec()\fP; they are -relevant only for \fBpcre[16|32]_exec()\fP without the JIT optimization. -. -. -.SS "Reducing \fBpcre[16|32]_exec()\fP's stack usage" -.rs -.sp -Each time that \fBmatch()\fP is actually called recursively, it uses memory -from the process stack. For certain kinds of pattern and data, very large -amounts of stack may be needed, despite the recognition of "tail recursion". -You can often reduce the amount of recursion, and therefore the amount of stack -used, by modifying the pattern that is being matched. Consider, for example, -this pattern: -.sp - ([^<]|<(?!inet))+ -.sp -It matches from wherever it starts until it encounters "<inet" or the end of -the data, and is the kind of pattern that might be used when processing an XML -file. Each iteration of the outer parentheses matches either one character that -is not "<" or a "<" that is not followed by "inet". However, each time a -parenthesis is processed, a recursion occurs, so this formulation uses a stack -frame for each matched character. For a long string, a lot of stack is -required. Consider now this rewritten pattern, which matches exactly the same -strings: -.sp - ([^<]++|<(?!inet))+ -.sp -This uses very much less stack, because runs of characters that do not contain -"<" are "swallowed" in one item inside the parentheses. Recursion happens only -when a "<" character that is not followed by "inet" is encountered (and we -assume this is relatively rare). A possessive quantifier is used to stop any -backtracking into the runs of non-"<" characters, but that is not related to -stack usage. -.P -This example shows that one way of avoiding stack problems when matching long -subject strings is to write repeated parenthesized subpatterns to match more -than one character whenever possible. -. -. -.SS "Compiling PCRE to use heap instead of stack for \fBpcre[16|32]_exec()\fP" -.rs -.sp -In environments where stack memory is constrained, you might want to compile -PCRE to use heap memory instead of stack for remembering back-up points when -\fBpcre[16|32]_exec()\fP is running. This makes it run a lot more slowly, however. -Details of how to do this are given in the -.\" HREF -\fBpcrebuild\fP -.\" -documentation. When built in this way, instead of using the stack, PCRE obtains -and frees memory by calling the functions that are pointed to by the -\fBpcre[16|32]_stack_malloc\fP and \fBpcre[16|32]_stack_free\fP variables. By -default, these point to \fBmalloc()\fP and \fBfree()\fP, but you can replace -the pointers to cause PCRE to use your own functions. Since the block sizes are -always the same, and are always freed in reverse order, it may be possible to -implement customized memory handlers that are more efficient than the standard -functions. -. -. -.SS "Limiting \fBpcre[16|32]_exec()\fP's stack usage" -.rs -.sp -You can set limits on the number of times that \fBmatch()\fP is called, both in -total and recursively. If a limit is exceeded, \fBpcre[16|32]_exec()\fP returns an -error code. Setting suitable limits should prevent it from running out of -stack. The default values of the limits are very large, and unlikely ever to -operate. They can be changed when PCRE is built, and they can also be set when -\fBpcre[16|32]_exec()\fP is called. For details of these interfaces, see the -.\" HREF -\fBpcrebuild\fP -.\" -documentation and the -.\" HTML <a href="pcreapi.html#extradata"> -.\" </a> -section on extra data for \fBpcre[16|32]_exec()\fP -.\" -in the -.\" HREF -\fBpcreapi\fP -.\" -documentation. -.P -As a very rough rule of thumb, you should reckon on about 500 bytes per -recursion. Thus, if you want to limit your stack usage to 8Mb, you should set -the limit at 16000 recursions. A 64Mb stack, on the other hand, can support -around 128000 recursions. -.P -In Unix-like environments, the \fBpcretest\fP test program has a command line -option (\fB-S\fP) that can be used to increase the size of its stack. As long -as the stack is large enough, another option (\fB-M\fP) can be used to find the -smallest limits that allow a particular pattern to match a given subject -string. This is done by calling \fBpcre[16|32]_exec()\fP repeatedly with different -limits. -. -. -.SS "Obtaining an estimate of stack usage" -.rs -.sp -The actual amount of stack used per recursion can vary quite a lot, depending -on the compiler that was used to build PCRE and the optimization or debugging -options that were set for it. The rule of thumb value of 500 bytes mentioned -above may be larger or smaller than what is actually needed. A better -approximation can be obtained by running this command: -.sp - pcretest -m -C -.sp -The \fB-C\fP option causes \fBpcretest\fP to output information about the -options with which PCRE was compiled. When \fB-m\fP is also given (before -\fB-C\fP), information about stack use is given in a line like this: -.sp - Match recursion uses stack: approximate frame size = 640 bytes -.sp -The value is approximate because some recursions need a bit more (up to perhaps -16 more bytes). -.P -If the above command is given when PCRE is compiled to use the heap instead of -the stack for recursion, the value that is output is the size of each block -that is obtained from the heap. -. -. -.SS "Changing stack size in Unix-like systems" -.rs -.sp -In Unix-like environments, there is not often a problem with the stack unless -very long strings are involved, though the default limit on stack size varies -from system to system. Values from 8Mb to 64Mb are common. You can find your -default limit by running the command: -.sp - ulimit -s -.sp -Unfortunately, the effect of running out of stack is often SIGSEGV, though -sometimes a more explicit error message is given. You can normally increase the -limit on stack size by code such as this: -.sp - struct rlimit rlim; - getrlimit(RLIMIT_STACK, &rlim); - rlim.rlim_cur = 100*1024*1024; - setrlimit(RLIMIT_STACK, &rlim); -.sp -This reads the current limits (soft and hard) using \fBgetrlimit()\fP, then -attempts to increase the soft limit to 100Mb using \fBsetrlimit()\fP. You must -do this before calling \fBpcre[16|32]_exec()\fP. -. -. -.SS "Changing stack size in Mac OS X" -.rs -.sp -Using \fBsetrlimit()\fP, as described above, should also work on Mac OS X. It -is also possible to set a stack size when linking a program. There is a -discussion about stack sizes in Mac OS X at this web site: -.\" HTML <a href="http://developer.apple.com/qa/qa2005/qa1419.html"> -.\" </a> -http://developer.apple.com/qa/qa2005/qa1419.html. -.\" -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 24 June 2012 -Copyright (c) 1997-2012 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcresyntax.3 b/plugins/Pcre16/docs/doc/pcresyntax.3 deleted file mode 100644 index fd878da4f9..0000000000 --- a/plugins/Pcre16/docs/doc/pcresyntax.3 +++ /dev/null @@ -1,517 +0,0 @@ -.TH PCRESYNTAX 3 "08 January 2014" "PCRE 8.35" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH "PCRE REGULAR EXPRESSION SYNTAX SUMMARY" -.rs -.sp -The full syntax and semantics of the regular expressions that are supported by -PCRE are described in the -.\" HREF -\fBpcrepattern\fP -.\" -documentation. This document contains a quick-reference summary of the syntax. -. -. -.SH "QUOTING" -.rs -.sp - \ex where x is non-alphanumeric is a literal x - \eQ...\eE treat enclosed characters as literal -. -. -.SH "CHARACTERS" -.rs -.sp - \ea alarm, that is, the BEL character (hex 07) - \ecx "control-x", where x is any ASCII character - \ee escape (hex 1B) - \ef form feed (hex 0C) - \en newline (hex 0A) - \er carriage return (hex 0D) - \et tab (hex 09) - \e0dd character with octal code 0dd - \eddd character with octal code ddd, or backreference - \eo{ddd..} character with octal code ddd.. - \exhh character with hex code hh - \ex{hhh..} character with hex code hhh.. -.sp -Note that \e0dd is always an octal code, and that \e8 and \e9 are the literal -characters "8" and "9". -. -. -.SH "CHARACTER TYPES" -.rs -.sp - . any character except newline; - in dotall mode, any character whatsoever - \eC one data unit, even in UTF mode (best avoided) - \ed a decimal digit - \eD a character that is not a decimal digit - \eh a horizontal white space character - \eH a character that is not a horizontal white space character - \eN a character that is not a newline - \ep{\fIxx\fP} a character with the \fIxx\fP property - \eP{\fIxx\fP} a character without the \fIxx\fP property - \eR a newline sequence - \es a white space character - \eS a character that is not a white space character - \ev a vertical white space character - \eV a character that is not a vertical white space character - \ew a "word" character - \eW a "non-word" character - \eX a Unicode extended grapheme cluster -.sp -By default, \ed, \es, and \ew match only ASCII characters, even in UTF-8 mode -or in the 16- bit and 32-bit libraries. However, if locale-specific matching is -happening, \es and \ew may also match characters with code points in the range -128-255. If the PCRE_UCP option is set, the behaviour of these escape sequences -is changed to use Unicode properties and they match many more characters. -. -. -.SH "GENERAL CATEGORY PROPERTIES FOR \ep and \eP" -.rs -.sp - C Other - Cc Control - Cf Format - Cn Unassigned - Co Private use - Cs Surrogate -.sp - L Letter - Ll Lower case letter - Lm Modifier letter - Lo Other letter - Lt Title case letter - Lu Upper case letter - L& Ll, Lu, or Lt -.sp - M Mark - Mc Spacing mark - Me Enclosing mark - Mn Non-spacing mark -.sp - N Number - Nd Decimal number - Nl Letter number - No Other number -.sp - P Punctuation - Pc Connector punctuation - Pd Dash punctuation - Pe Close punctuation - Pf Final punctuation - Pi Initial punctuation - Po Other punctuation - Ps Open punctuation -.sp - S Symbol - Sc Currency symbol - Sk Modifier symbol - Sm Mathematical symbol - So Other symbol -.sp - Z Separator - Zl Line separator - Zp Paragraph separator - Zs Space separator -. -. -.SH "PCRE SPECIAL CATEGORY PROPERTIES FOR \ep and \eP" -.rs -.sp - Xan Alphanumeric: union of properties L and N - Xps POSIX space: property Z or tab, NL, VT, FF, CR - Xsp Perl space: property Z or tab, NL, VT, FF, CR - Xuc Univerally-named character: one that can be - represented by a Universal Character Name - Xwd Perl word: property Xan or underscore -.sp -Perl and POSIX space are now the same. Perl added VT to its space character set -at release 5.18 and PCRE changed at release 8.34. -. -. -.SH "SCRIPT NAMES FOR \ep AND \eP" -.rs -.sp -Arabic, -Armenian, -Avestan, -Balinese, -Bamum, -Batak, -Bengali, -Bopomofo, -Brahmi, -Braille, -Buginese, -Buhid, -Canadian_Aboriginal, -Carian, -Chakma, -Cham, -Cherokee, -Common, -Coptic, -Cuneiform, -Cypriot, -Cyrillic, -Deseret, -Devanagari, -Egyptian_Hieroglyphs, -Ethiopic, -Georgian, -Glagolitic, -Gothic, -Greek, -Gujarati, -Gurmukhi, -Han, -Hangul, -Hanunoo, -Hebrew, -Hiragana, -Imperial_Aramaic, -Inherited, -Inscriptional_Pahlavi, -Inscriptional_Parthian, -Javanese, -Kaithi, -Kannada, -Katakana, -Kayah_Li, -Kharoshthi, -Khmer, -Lao, -Latin, -Lepcha, -Limbu, -Linear_B, -Lisu, -Lycian, -Lydian, -Malayalam, -Mandaic, -Meetei_Mayek, -Meroitic_Cursive, -Meroitic_Hieroglyphs, -Miao, -Mongolian, -Myanmar, -New_Tai_Lue, -Nko, -Ogham, -Old_Italic, -Old_Persian, -Old_South_Arabian, -Old_Turkic, -Ol_Chiki, -Oriya, -Osmanya, -Phags_Pa, -Phoenician, -Rejang, -Runic, -Samaritan, -Saurashtra, -Sharada, -Shavian, -Sinhala, -Sora_Sompeng, -Sundanese, -Syloti_Nagri, -Syriac, -Tagalog, -Tagbanwa, -Tai_Le, -Tai_Tham, -Tai_Viet, -Takri, -Tamil, -Telugu, -Thaana, -Thai, -Tibetan, -Tifinagh, -Ugaritic, -Vai, -Yi. -. -. -.SH "CHARACTER CLASSES" -.rs -.sp - [...] positive character class - [^...] negative character class - [x-y] range (can be used for hex characters) - [[:xxx:]] positive POSIX named set - [[:^xxx:]] negative POSIX named set -.sp - alnum alphanumeric - alpha alphabetic - ascii 0-127 - blank space or tab - cntrl control character - digit decimal digit - graph printing, excluding space - lower lower case letter - print printing, including space - punct printing, excluding alphanumeric - space white space - upper upper case letter - word same as \ew - xdigit hexadecimal digit -.sp -In PCRE, POSIX character set names recognize only ASCII characters by default, -but some of them use Unicode properties if PCRE_UCP is set. You can use -\eQ...\eE inside a character class. -. -. -.SH "QUANTIFIERS" -.rs -.sp - ? 0 or 1, greedy - ?+ 0 or 1, possessive - ?? 0 or 1, lazy - * 0 or more, greedy - *+ 0 or more, possessive - *? 0 or more, lazy - + 1 or more, greedy - ++ 1 or more, possessive - +? 1 or more, lazy - {n} exactly n - {n,m} at least n, no more than m, greedy - {n,m}+ at least n, no more than m, possessive - {n,m}? at least n, no more than m, lazy - {n,} n or more, greedy - {n,}+ n or more, possessive - {n,}? n or more, lazy -. -. -.SH "ANCHORS AND SIMPLE ASSERTIONS" -.rs -.sp - \eb word boundary - \eB not a word boundary - ^ start of subject - also after internal newline in multiline mode - \eA start of subject - $ end of subject - also before newline at end of subject - also before internal newline in multiline mode - \eZ end of subject - also before newline at end of subject - \ez end of subject - \eG first matching position in subject -. -. -.SH "MATCH POINT RESET" -.rs -.sp - \eK reset start of match -.sp -\eK is honoured in positive assertions, but ignored in negative ones. -. -. -.SH "ALTERNATION" -.rs -.sp - expr|expr|expr... -. -. -.SH "CAPTURING" -.rs -.sp - (...) capturing group - (?<name>...) named capturing group (Perl) - (?'name'...) named capturing group (Perl) - (?P<name>...) named capturing group (Python) - (?:...) non-capturing group - (?|...) non-capturing group; reset group numbers for - capturing groups in each alternative -. -. -.SH "ATOMIC GROUPS" -.rs -.sp - (?>...) atomic, non-capturing group -. -. -. -. -.SH "COMMENT" -.rs -.sp - (?#....) comment (not nestable) -. -. -.SH "OPTION SETTING" -.rs -.sp - (?i) caseless - (?J) allow duplicate names - (?m) multiline - (?s) single line (dotall) - (?U) default ungreedy (lazy) - (?x) extended (ignore white space) - (?-...) unset option(s) -.sp -The following are recognized only at the very start of a pattern or after one -of the newline or \eR options with similar syntax. More than one of them may -appear. -.sp - (*LIMIT_MATCH=d) set the match limit to d (decimal number) - (*LIMIT_RECURSION=d) set the recursion limit to d (decimal number) - (*NO_AUTO_POSSESS) no auto-possessification (PCRE_NO_AUTO_POSSESS) - (*NO_START_OPT) no start-match optimization (PCRE_NO_START_OPTIMIZE) - (*UTF8) set UTF-8 mode: 8-bit library (PCRE_UTF8) - (*UTF16) set UTF-16 mode: 16-bit library (PCRE_UTF16) - (*UTF32) set UTF-32 mode: 32-bit library (PCRE_UTF32) - (*UTF) set appropriate UTF mode for the library in use - (*UCP) set PCRE_UCP (use Unicode properties for \ed etc) -.sp -Note that LIMIT_MATCH and LIMIT_RECURSION can only reduce the value of the -limits set by the caller of pcre_exec(), not increase them. -. -. -.SH "NEWLINE CONVENTION" -.rs -.sp -These are recognized only at the very start of the pattern or after option -settings with a similar syntax. -.sp - (*CR) carriage return only - (*LF) linefeed only - (*CRLF) carriage return followed by linefeed - (*ANYCRLF) all three of the above - (*ANY) any Unicode newline sequence -. -. -.SH "WHAT \eR MATCHES" -.rs -.sp -These are recognized only at the very start of the pattern or after option -setting with a similar syntax. -.sp - (*BSR_ANYCRLF) CR, LF, or CRLF - (*BSR_UNICODE) any Unicode newline sequence -. -. -.SH "LOOKAHEAD AND LOOKBEHIND ASSERTIONS" -.rs -.sp - (?=...) positive look ahead - (?!...) negative look ahead - (?<=...) positive look behind - (?<!...) negative look behind -.sp -Each top-level branch of a look behind must be of a fixed length. -. -. -.SH "BACKREFERENCES" -.rs -.sp - \en reference by number (can be ambiguous) - \egn reference by number - \eg{n} reference by number - \eg{-n} relative reference by number - \ek<name> reference by name (Perl) - \ek'name' reference by name (Perl) - \eg{name} reference by name (Perl) - \ek{name} reference by name (.NET) - (?P=name) reference by name (Python) -. -. -.SH "SUBROUTINE REFERENCES (POSSIBLY RECURSIVE)" -.rs -.sp - (?R) recurse whole pattern - (?n) call subpattern by absolute number - (?+n) call subpattern by relative number - (?-n) call subpattern by relative number - (?&name) call subpattern by name (Perl) - (?P>name) call subpattern by name (Python) - \eg<name> call subpattern by name (Oniguruma) - \eg'name' call subpattern by name (Oniguruma) - \eg<n> call subpattern by absolute number (Oniguruma) - \eg'n' call subpattern by absolute number (Oniguruma) - \eg<+n> call subpattern by relative number (PCRE extension) - \eg'+n' call subpattern by relative number (PCRE extension) - \eg<-n> call subpattern by relative number (PCRE extension) - \eg'-n' call subpattern by relative number (PCRE extension) -. -. -.SH "CONDITIONAL PATTERNS" -.rs -.sp - (?(condition)yes-pattern) - (?(condition)yes-pattern|no-pattern) -.sp - (?(n)... absolute reference condition - (?(+n)... relative reference condition - (?(-n)... relative reference condition - (?(<name>)... named reference condition (Perl) - (?('name')... named reference condition (Perl) - (?(name)... named reference condition (PCRE) - (?(R)... overall recursion condition - (?(Rn)... specific group recursion condition - (?(R&name)... specific recursion condition - (?(DEFINE)... define subpattern for reference - (?(assert)... assertion condition -. -. -.SH "BACKTRACKING CONTROL" -.rs -.sp -The following act immediately they are reached: -.sp - (*ACCEPT) force successful match - (*FAIL) force backtrack; synonym (*F) - (*MARK:NAME) set name to be passed back; synonym (*:NAME) -.sp -The following act only when a subsequent match failure causes a backtrack to -reach them. They all force a match failure, but they differ in what happens -afterwards. Those that advance the start-of-match point do so only if the -pattern is not anchored. -.sp - (*COMMIT) overall failure, no advance of starting point - (*PRUNE) advance to next starting character - (*PRUNE:NAME) equivalent to (*MARK:NAME)(*PRUNE) - (*SKIP) advance to current matching position - (*SKIP:NAME) advance to position corresponding to an earlier - (*MARK:NAME); if not found, the (*SKIP) is ignored - (*THEN) local failure, backtrack to next alternation - (*THEN:NAME) equivalent to (*MARK:NAME)(*THEN) -. -. -.SH "CALLOUTS" -.rs -.sp - (?C) callout - (?Cn) callout with data n -. -. -.SH "SEE ALSO" -.rs -.sp -\fBpcrepattern\fP(3), \fBpcreapi\fP(3), \fBpcrecallout\fP(3), -\fBpcrematching\fP(3), \fBpcre\fP(3). -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 08 January 2014 -Copyright (c) 1997-2014 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcretest.1 b/plugins/Pcre16/docs/doc/pcretest.1 deleted file mode 100644 index 92640da8e1..0000000000 --- a/plugins/Pcre16/docs/doc/pcretest.1 +++ /dev/null @@ -1,1156 +0,0 @@ -.TH PCRETEST 1 "09 February 2014" "PCRE 8.35" -.SH NAME -pcretest - a program for testing Perl-compatible regular expressions. -.SH SYNOPSIS -.rs -.sp -.B pcretest "[options] [input file [output file]]" -.sp -\fBpcretest\fP was written as a test program for the PCRE regular expression -library itself, but it can also be used for experimenting with regular -expressions. This document describes the features of the test program; for -details of the regular expressions themselves, see the -.\" HREF -\fBpcrepattern\fP -.\" -documentation. For details of the PCRE library function calls and their -options, see the -.\" HREF -\fBpcreapi\fP -.\" -, -.\" HREF -\fBpcre16\fP -and -.\" HREF -\fBpcre32\fP -.\" -documentation. -.P -The input for \fBpcretest\fP is a sequence of regular expression patterns and -strings to be matched, as described below. The output shows the result of each -match. Options on the command line and the patterns control PCRE options and -exactly what is output. -.P -As PCRE has evolved, it has acquired many different features, and as a result, -\fBpcretest\fP now has rather a lot of obscure options for testing every -possible feature. Some of these options are specifically designed for use in -conjunction with the test script and data files that are distributed as part of -PCRE, and are unlikely to be of use otherwise. They are all documented here, -but without much justification. -. -. -.SH "INPUT DATA FORMAT" -.rs -.sp -Input to \fBpcretest\fP is processed line by line, either by calling the C -library's \fBfgets()\fP function, or via the \fBlibreadline\fP library (see -below). In Unix-like environments, \fBfgets()\fP treats any bytes other than -newline as data characters. However, in some Windows environments character 26 -(hex 1A) causes an immediate end of file, and no further data is read. For -maximum portability, therefore, it is safest to use only ASCII characters in -\fBpcretest\fP input files. -. -. -.SH "PCRE's 8-BIT, 16-BIT AND 32-BIT LIBRARIES" -.rs -.sp -From release 8.30, two separate PCRE libraries can be built. The original one -supports 8-bit character strings, whereas the newer 16-bit library supports -character strings encoded in 16-bit units. From release 8.32, a third library -can be built, supporting character strings encoded in 32-bit units. The -\fBpcretest\fP program can be used to test all three libraries. However, it is -itself still an 8-bit program, reading 8-bit input and writing 8-bit output. -When testing the 16-bit or 32-bit library, the patterns and data strings are -converted to 16- or 32-bit format before being passed to the PCRE library -functions. Results are converted to 8-bit for output. -.P -References to functions and structures of the form \fBpcre[16|32]_xx\fP below -mean "\fBpcre_xx\fP when using the 8-bit library, \fBpcre16_xx\fP when using -the 16-bit library, or \fBpcre32_xx\fP when using the 32-bit library". -. -. -.SH "COMMAND LINE OPTIONS" -.rs -.TP 10 -\fB-8\fP -If both the 8-bit library has been built, this option causes the 8-bit library -to be used (which is the default); if the 8-bit library has not been built, -this option causes an error. -.TP 10 -\fB-16\fP -If both the 8-bit or the 32-bit, and the 16-bit libraries have been built, this -option causes the 16-bit library to be used. If only the 16-bit library has been -built, this is the default (so has no effect). If only the 8-bit or the 32-bit -library has been built, this option causes an error. -.TP 10 -\fB-32\fP -If both the 8-bit or the 16-bit, and the 32-bit libraries have been built, this -option causes the 32-bit library to be used. If only the 32-bit library has been -built, this is the default (so has no effect). If only the 8-bit or the 16-bit -library has been built, this option causes an error. -.TP 10 -\fB-b\fP -Behave as if each pattern has the \fB/B\fP (show byte code) modifier; the -internal form is output after compilation. -.TP 10 -\fB-C\fP -Output the version number of the PCRE library, and all available information -about the optional features that are included, and then exit with zero exit -code. All other options are ignored. -.TP 10 -\fB-C\fP \fIoption\fP -Output information about a specific build-time option, then exit. This -functionality is intended for use in scripts such as \fBRunTest\fP. The -following options output the value and set the exit code as indicated: -.sp - ebcdic-nl the code for LF (= NL) in an EBCDIC environment: - 0x15 or 0x25 - 0 if used in an ASCII environment - exit code is always 0 - linksize the configured internal link size (2, 3, or 4) - exit code is set to the link size - newline the default newline setting: - CR, LF, CRLF, ANYCRLF, or ANY - exit code is always 0 - bsr the default setting for what \eR matches: - ANYCRLF or ANY - exit code is always 0 -.sp -The following options output 1 for true or 0 for false, and set the exit code -to the same value: -.sp - ebcdic compiled for an EBCDIC environment - jit just-in-time support is available - pcre16 the 16-bit library was built - pcre32 the 32-bit library was built - pcre8 the 8-bit library was built - ucp Unicode property support is available - utf UTF-8 and/or UTF-16 and/or UTF-32 support - is available -.sp -If an unknown option is given, an error message is output; the exit code is 0. -.TP 10 -\fB-d\fP -Behave as if each pattern has the \fB/D\fP (debug) modifier; the internal -form and information about the compiled pattern is output after compilation; -\fB-d\fP is equivalent to \fB-b -i\fP. -.TP 10 -\fB-dfa\fP -Behave as if each data line contains the \eD escape sequence; this causes the -alternative matching function, \fBpcre[16|32]_dfa_exec()\fP, to be used instead -of the standard \fBpcre[16|32]_exec()\fP function (more detail is given below). -.TP 10 -\fB-help\fP -Output a brief summary these options and then exit. -.TP 10 -\fB-i\fP -Behave as if each pattern has the \fB/I\fP modifier; information about the -compiled pattern is given after compilation. -.TP 10 -\fB-M\fP -Behave as if each data line contains the \eM escape sequence; this causes -PCRE to discover the minimum MATCH_LIMIT and MATCH_LIMIT_RECURSION settings by -calling \fBpcre[16|32]_exec()\fP repeatedly with different limits. -.TP 10 -\fB-m\fP -Output the size of each compiled pattern after it has been compiled. This is -equivalent to adding \fB/M\fP to each regular expression. The size is given in -bytes for both libraries. -.TP 10 -\fB-O\fP -Behave as if each pattern has the \fB/O\fP modifier, that is disable -auto-possessification for all patterns. -.TP 10 -\fB-o\fP \fIosize\fP -Set the number of elements in the output vector that is used when calling -\fBpcre[16|32]_exec()\fP or \fBpcre[16|32]_dfa_exec()\fP to be \fIosize\fP. The -default value is 45, which is enough for 14 capturing subexpressions for -\fBpcre[16|32]_exec()\fP or 22 different matches for -\fBpcre[16|32]_dfa_exec()\fP. -The vector size can be changed for individual matching calls by including \eO -in the data line (see below). -.TP 10 -\fB-p\fP -Behave as if each pattern has the \fB/P\fP modifier; the POSIX wrapper API is -used to call PCRE. None of the other options has any effect when \fB-p\fP is -set. This option can be used only with the 8-bit library. -.TP 10 -\fB-q\fP -Do not output the version number of \fBpcretest\fP at the start of execution. -.TP 10 -\fB-S\fP \fIsize\fP -On Unix-like systems, set the size of the run-time stack to \fIsize\fP -megabytes. -.TP 10 -\fB-s\fP or \fB-s+\fP -Behave as if each pattern has the \fB/S\fP modifier; in other words, force each -pattern to be studied. If \fB-s+\fP is used, all the JIT compile options are -passed to \fBpcre[16|32]_study()\fP, causing just-in-time optimization to be set -up if it is available, for both full and partial matching. Specific JIT compile -options can be selected by following \fB-s+\fP with a digit in the range 1 to -7, which selects the JIT compile modes as follows: -.sp - 1 normal match only - 2 soft partial match only - 3 normal match and soft partial match - 4 hard partial match only - 6 soft and hard partial match - 7 all three modes (default) -.sp -If \fB-s++\fP is used instead of \fB-s+\fP (with or without a following digit), -the text "(JIT)" is added to the first output line after a match or no match -when JIT-compiled code was actually used. -.sp -Note that there are pattern options that can override \fB-s\fP, either -specifying no studying at all, or suppressing JIT compilation. -.sp -If the \fB/I\fP or \fB/D\fP option is present on a pattern (requesting output -about the compiled pattern), information about the result of studying is not -included when studying is caused only by \fB-s\fP and neither \fB-i\fP nor -\fB-d\fP is present on the command line. This behaviour means that the output -from tests that are run with and without \fB-s\fP should be identical, except -when options that output information about the actual running of a match are -set. -.sp -The \fB-M\fP, \fB-t\fP, and \fB-tm\fP options, which give information about -resources used, are likely to produce different output with and without -\fB-s\fP. Output may also differ if the \fB/C\fP option is present on an -individual pattern. This uses callouts to trace the the matching process, and -this may be different between studied and non-studied patterns. If the pattern -contains (*MARK) items there may also be differences, for the same reason. The -\fB-s\fP command line option can be overridden for specific patterns that -should never be studied (see the \fB/S\fP pattern modifier below). -.TP 10 -\fB-t\fP -Run each compile, study, and match many times with a timer, and output the -resulting times per compile, study, or match (in milliseconds). Do not set -\fB-m\fP with \fB-t\fP, because you will then get the size output a zillion -times, and the timing will be distorted. You can control the number of -iterations that are used for timing by following \fB-t\fP with a number (as a -separate item on the command line). For example, "-t 1000" iterates 1000 times. -The default is to iterate 500000 times. -.TP 10 -\fB-tm\fP -This is like \fB-t\fP except that it times only the matching phase, not the -compile or study phases. -.TP 10 -\fB-T\fP \fB-TM\fP -These behave like \fB-t\fP and \fB-tm\fP, but in addition, at the end of a run, -the total times for all compiles, studies, and matches are output. -. -. -.SH DESCRIPTION -.rs -.sp -If \fBpcretest\fP is given two filename arguments, it reads from the first and -writes to the second. If it is given only one filename argument, it reads from -that file and writes to stdout. Otherwise, it reads from stdin and writes to -stdout, and prompts for each line of input, using "re>" to prompt for regular -expressions, and "data>" to prompt for data lines. -.P -When \fBpcretest\fP is built, a configuration option can specify that it should -be linked with the \fBlibreadline\fP library. When this is done, if the input -is from a terminal, it is read using the \fBreadline()\fP function. This -provides line-editing and history facilities. The output from the \fB-help\fP -option states whether or not \fBreadline()\fP will be used. -.P -The program handles any number of sets of input on a single input file. Each -set starts with a regular expression, and continues with any number of data -lines to be matched against that pattern. -.P -Each data line is matched separately and independently. If you want to do -multi-line matches, you have to use the \en escape sequence (or \er or \er\en, -etc., depending on the newline setting) in a single line of input to encode the -newline sequences. There is no limit on the length of data lines; the input -buffer is automatically extended if it is too small. -.P -An empty line signals the end of the data lines, at which point a new regular -expression is read. The regular expressions are given enclosed in any -non-alphanumeric delimiters other than backslash, for example: -.sp - /(a|bc)x+yz/ -.sp -White space before the initial delimiter is ignored. A regular expression may -be continued over several input lines, in which case the newline characters are -included within it. It is possible to include the delimiter within the pattern -by escaping it, for example -.sp - /abc\e/def/ -.sp -If you do so, the escape and the delimiter form part of the pattern, but since -delimiters are always non-alphanumeric, this does not affect its interpretation. -If the terminating delimiter is immediately followed by a backslash, for -example, -.sp - /abc/\e -.sp -then a backslash is added to the end of the pattern. This is done to provide a -way of testing the error condition that arises if a pattern finishes with a -backslash, because -.sp - /abc\e/ -.sp -is interpreted as the first line of a pattern that starts with "abc/", causing -pcretest to read the next line as a continuation of the regular expression. -. -. -.SH "PATTERN MODIFIERS" -.rs -.sp -A pattern may be followed by any number of modifiers, which are mostly single -characters, though some of these can be qualified by further characters. -Following Perl usage, these are referred to below as, for example, "the -\fB/i\fP modifier", even though the delimiter of the pattern need not always be -a slash, and no slash is used when writing modifiers. White space may appear -between the final pattern delimiter and the first modifier, and between the -modifiers themselves. For reference, here is a complete list of modifiers. They -fall into several groups that are described in detail in the following -sections. -.sp - \fB/8\fP set UTF mode - \fB/9\fP set PCRE_NEVER_UTF (locks out UTF mode) - \fB/?\fP disable UTF validity check - \fB/+\fP show remainder of subject after match - \fB/=\fP show all captures (not just those that are set) -.sp - \fB/A\fP set PCRE_ANCHORED - \fB/B\fP show compiled code - \fB/C\fP set PCRE_AUTO_CALLOUT - \fB/D\fP same as \fB/B\fP plus \fB/I\fP - \fB/E\fP set PCRE_DOLLAR_ENDONLY - \fB/F\fP flip byte order in compiled pattern - \fB/f\fP set PCRE_FIRSTLINE - \fB/G\fP find all matches (shorten string) - \fB/g\fP find all matches (use startoffset) - \fB/I\fP show information about pattern - \fB/i\fP set PCRE_CASELESS - \fB/J\fP set PCRE_DUPNAMES - \fB/K\fP show backtracking control names - \fB/L\fP set locale - \fB/M\fP show compiled memory size - \fB/m\fP set PCRE_MULTILINE - \fB/N\fP set PCRE_NO_AUTO_CAPTURE - \fB/O\fP set PCRE_NO_AUTO_POSSESS - \fB/P\fP use the POSIX wrapper - \fB/Q\fP test external stack check function - \fB/S\fP study the pattern after compilation - \fB/s\fP set PCRE_DOTALL - \fB/T\fP select character tables - \fB/U\fP set PCRE_UNGREEDY - \fB/W\fP set PCRE_UCP - \fB/X\fP set PCRE_EXTRA - \fB/x\fP set PCRE_EXTENDED - \fB/Y\fP set PCRE_NO_START_OPTIMIZE - \fB/Z\fP don't show lengths in \fB/B\fP output -.sp - \fB/<any>\fP set PCRE_NEWLINE_ANY - \fB/<anycrlf>\fP set PCRE_NEWLINE_ANYCRLF - \fB/<cr>\fP set PCRE_NEWLINE_CR - \fB/<crlf>\fP set PCRE_NEWLINE_CRLF - \fB/<lf>\fP set PCRE_NEWLINE_LF - \fB/<bsr_anycrlf>\fP set PCRE_BSR_ANYCRLF - \fB/<bsr_unicode>\fP set PCRE_BSR_UNICODE - \fB/<JS>\fP set PCRE_JAVASCRIPT_COMPAT -.sp -. -. -.SS "Perl-compatible modifiers" -.rs -.sp -The \fB/i\fP, \fB/m\fP, \fB/s\fP, and \fB/x\fP modifiers set the PCRE_CASELESS, -PCRE_MULTILINE, PCRE_DOTALL, or PCRE_EXTENDED options, respectively, when -\fBpcre[16|32]_compile()\fP is called. These four modifier letters have the same -effect as they do in Perl. For example: -.sp - /caseless/i -.sp -. -. -.SS "Modifiers for other PCRE options" -.rs -.sp -The following table shows additional modifiers for setting PCRE compile-time -options that do not correspond to anything in Perl: -.sp - \fB/8\fP PCRE_UTF8 ) when using the 8-bit - \fB/?\fP PCRE_NO_UTF8_CHECK ) library -.sp - \fB/8\fP PCRE_UTF16 ) when using the 16-bit - \fB/?\fP PCRE_NO_UTF16_CHECK ) library -.sp - \fB/8\fP PCRE_UTF32 ) when using the 32-bit - \fB/?\fP PCRE_NO_UTF32_CHECK ) library -.sp - \fB/9\fP PCRE_NEVER_UTF - \fB/A\fP PCRE_ANCHORED - \fB/C\fP PCRE_AUTO_CALLOUT - \fB/E\fP PCRE_DOLLAR_ENDONLY - \fB/f\fP PCRE_FIRSTLINE - \fB/J\fP PCRE_DUPNAMES - \fB/N\fP PCRE_NO_AUTO_CAPTURE - \fB/O\fP PCRE_NO_AUTO_POSSESS - \fB/U\fP PCRE_UNGREEDY - \fB/W\fP PCRE_UCP - \fB/X\fP PCRE_EXTRA - \fB/Y\fP PCRE_NO_START_OPTIMIZE - \fB/<any>\fP PCRE_NEWLINE_ANY - \fB/<anycrlf>\fP PCRE_NEWLINE_ANYCRLF - \fB/<cr>\fP PCRE_NEWLINE_CR - \fB/<crlf>\fP PCRE_NEWLINE_CRLF - \fB/<lf>\fP PCRE_NEWLINE_LF - \fB/<bsr_anycrlf>\fP PCRE_BSR_ANYCRLF - \fB/<bsr_unicode>\fP PCRE_BSR_UNICODE - \fB/<JS>\fP PCRE_JAVASCRIPT_COMPAT -.sp -The modifiers that are enclosed in angle brackets are literal strings as shown, -including the angle brackets, but the letters within can be in either case. -This example sets multiline matching with CRLF as the line ending sequence: -.sp - /^abc/m<CRLF> -.sp -As well as turning on the PCRE_UTF8/16/32 option, the \fB/8\fP modifier causes -all non-printing characters in output strings to be printed using the -\ex{hh...} notation. Otherwise, those less than 0x100 are output in hex without -the curly brackets. -.P -Full details of the PCRE options are given in the -.\" HREF -\fBpcreapi\fP -.\" -documentation. -. -. -.SS "Finding all matches in a string" -.rs -.sp -Searching for all possible matches within each subject string can be requested -by the \fB/g\fP or \fB/G\fP modifier. After finding a match, PCRE is called -again to search the remainder of the subject string. The difference between -\fB/g\fP and \fB/G\fP is that the former uses the \fIstartoffset\fP argument to -\fBpcre[16|32]_exec()\fP to start searching at a new point within the entire -string (which is in effect what Perl does), whereas the latter passes over a -shortened substring. This makes a difference to the matching process if the -pattern begins with a lookbehind assertion (including \eb or \eB). -.P -If any call to \fBpcre[16|32]_exec()\fP in a \fB/g\fP or \fB/G\fP sequence matches -an empty string, the next call is done with the PCRE_NOTEMPTY_ATSTART and -PCRE_ANCHORED flags set in order to search for another, non-empty, match at the -same point. If this second match fails, the start offset is advanced, and the -normal match is retried. This imitates the way Perl handles such cases when -using the \fB/g\fP modifier or the \fBsplit()\fP function. Normally, the start -offset is advanced by one character, but if the newline convention recognizes -CRLF as a newline, and the current character is CR followed by LF, an advance -of two is used. -. -. -.SS "Other modifiers" -.rs -.sp -There are yet more modifiers for controlling the way \fBpcretest\fP -operates. -.P -The \fB/+\fP modifier requests that as well as outputting the substring that -matched the entire pattern, \fBpcretest\fP should in addition output the -remainder of the subject string. This is useful for tests where the subject -contains multiple copies of the same substring. If the \fB+\fP modifier appears -twice, the same action is taken for captured substrings. In each case the -remainder is output on the following line with a plus character following the -capture number. Note that this modifier must not immediately follow the /S -modifier because /S+ and /S++ have other meanings. -.P -The \fB/=\fP modifier requests that the values of all potential captured -parentheses be output after a match. By default, only those up to the highest -one actually used in the match are output (corresponding to the return code -from \fBpcre[16|32]_exec()\fP). Values in the offsets vector corresponding to -higher numbers should be set to -1, and these are output as "<unset>". This -modifier gives a way of checking that this is happening. -.P -The \fB/B\fP modifier is a debugging feature. It requests that \fBpcretest\fP -output a representation of the compiled code after compilation. Normally this -information contains length and offset values; however, if \fB/Z\fP is also -present, this data is replaced by spaces. This is a special feature for use in -the automatic test scripts; it ensures that the same output is generated for -different internal link sizes. -.P -The \fB/D\fP modifier is a PCRE debugging feature, and is equivalent to -\fB/BI\fP, that is, both the \fB/B\fP and the \fB/I\fP modifiers. -.P -The \fB/F\fP modifier causes \fBpcretest\fP to flip the byte order of the -2-byte and 4-byte fields in the compiled pattern. This facility is for testing -the feature in PCRE that allows it to execute patterns that were compiled on a -host with a different endianness. This feature is not available when the POSIX -interface to PCRE is being used, that is, when the \fB/P\fP pattern modifier is -specified. See also the section about saving and reloading compiled patterns -below. -.P -The \fB/I\fP modifier requests that \fBpcretest\fP output information about the -compiled pattern (whether it is anchored, has a fixed first character, and -so on). It does this by calling \fBpcre[16|32]_fullinfo()\fP after compiling a -pattern. If the pattern is studied, the results of that are also output. In -this output, the word "char" means a non-UTF character, that is, the value of a -single data item (8-bit, 16-bit, or 32-bit, depending on the library that is -being tested). -.P -The \fB/K\fP modifier requests \fBpcretest\fP to show names from backtracking -control verbs that are returned from calls to \fBpcre[16|32]_exec()\fP. It causes -\fBpcretest\fP to create a \fBpcre[16|32]_extra\fP block if one has not already -been created by a call to \fBpcre[16|32]_study()\fP, and to set the -PCRE_EXTRA_MARK flag and the \fBmark\fP field within it, every time that -\fBpcre[16|32]_exec()\fP is called. If the variable that the \fBmark\fP field -points to is non-NULL for a match, non-match, or partial match, \fBpcretest\fP -prints the string to which it points. For a match, this is shown on a line by -itself, tagged with "MK:". For a non-match it is added to the message. -.P -The \fB/L\fP modifier must be followed directly by the name of a locale, for -example, -.sp - /pattern/Lfr_FR -.sp -For this reason, it must be the last modifier. The given locale is set, -\fBpcre[16|32]_maketables()\fP is called to build a set of character tables for -the locale, and this is then passed to \fBpcre[16|32]_compile()\fP when compiling -the regular expression. Without an \fB/L\fP (or \fB/T\fP) modifier, NULL is -passed as the tables pointer; that is, \fB/L\fP applies only to the expression -on which it appears. -.P -The \fB/M\fP modifier causes the size in bytes of the memory block used to hold -the compiled pattern to be output. This does not include the size of the -\fBpcre[16|32]\fP block; it is just the actual compiled data. If the pattern is -successfully studied with the PCRE_STUDY_JIT_COMPILE option, the size of the -JIT compiled code is also output. -.P -The \fB/Q\fP modifier is used to test the use of \fBpcre_stack_guard\fP. It -must be followed by '0' or '1', specifying the return code to be given from an -external function that is passed to PCRE and used for stack checking during -compilation (see the -.\" HREF -\fBpcreapi\fP -.\" -documentation for details). -.P -The \fB/S\fP modifier causes \fBpcre[16|32]_study()\fP to be called after the -expression has been compiled, and the results used when the expression is -matched. There are a number of qualifying characters that may follow \fB/S\fP. -They may appear in any order. -.P -If \fB/S\fP is followed by an exclamation mark, \fBpcre[16|32]_study()\fP is -called with the PCRE_STUDY_EXTRA_NEEDED option, causing it always to return a -\fBpcre_extra\fP block, even when studying discovers no useful information. -.P -If \fB/S\fP is followed by a second S character, it suppresses studying, even -if it was requested externally by the \fB-s\fP command line option. This makes -it possible to specify that certain patterns are always studied, and others are -never studied, independently of \fB-s\fP. This feature is used in the test -files in a few cases where the output is different when the pattern is studied. -.P -If the \fB/S\fP modifier is followed by a + character, the call to -\fBpcre[16|32]_study()\fP is made with all the JIT study options, requesting -just-in-time optimization support if it is available, for both normal and -partial matching. If you want to restrict the JIT compiling modes, you can -follow \fB/S+\fP with a digit in the range 1 to 7: -.sp - 1 normal match only - 2 soft partial match only - 3 normal match and soft partial match - 4 hard partial match only - 6 soft and hard partial match - 7 all three modes (default) -.sp -If \fB/S++\fP is used instead of \fB/S+\fP (with or without a following digit), -the text "(JIT)" is added to the first output line after a match or no match -when JIT-compiled code was actually used. -.P -Note that there is also an independent \fB/+\fP modifier; it must not be given -immediately after \fB/S\fP or \fB/S+\fP because this will be misinterpreted. -.P -If JIT studying is successful, the compiled JIT code will automatically be used -when \fBpcre[16|32]_exec()\fP is run, except when incompatible run-time options -are specified. For more details, see the -.\" HREF -\fBpcrejit\fP -.\" -documentation. See also the \fB\eJ\fP escape sequence below for a way of -setting the size of the JIT stack. -.P -Finally, if \fB/S\fP is followed by a minus character, JIT compilation is -suppressed, even if it was requested externally by the \fB-s\fP command line -option. This makes it possible to specify that JIT is never to be used for -certain patterns. -.P -The \fB/T\fP modifier must be followed by a single digit. It causes a specific -set of built-in character tables to be passed to \fBpcre[16|32]_compile()\fP. It -is used in the standard PCRE tests to check behaviour with different character -tables. The digit specifies the tables as follows: -.sp - 0 the default ASCII tables, as distributed in - pcre_chartables.c.dist - 1 a set of tables defining ISO 8859 characters -.sp -In table 1, some characters whose codes are greater than 128 are identified as -letters, digits, spaces, etc. -. -. -.SS "Using the POSIX wrapper API" -.rs -.sp -The \fB/P\fP modifier causes \fBpcretest\fP to call PCRE via the POSIX wrapper -API rather than its native API. This supports only the 8-bit library. When -\fB/P\fP is set, the following modifiers set options for the \fBregcomp()\fP -function: -.sp - /i REG_ICASE - /m REG_NEWLINE - /N REG_NOSUB - /s REG_DOTALL ) - /U REG_UNGREEDY ) These options are not part of - /W REG_UCP ) the POSIX standard - /8 REG_UTF8 ) -.sp -The \fB/+\fP modifier works as described above. All other modifiers are -ignored. -. -. -.SS "Locking out certain modifiers" -.rs -.sp -PCRE can be compiled with or without support for certain features such as -UTF-8/16/32 or Unicode properties. Accordingly, the standard tests are split up -into a number of different files that are selected for running depending on -which features are available. When updating the tests, it is all too easy to -put a new test into the wrong file by mistake; for example, to put a test that -requires UTF support into a file that is used when it is not available. To help -detect such mistakes as early as possible, there is a facility for locking out -specific modifiers. If an input line for \fBpcretest\fP starts with the string -"< forbid " the following sequence of characters is taken as a list of -forbidden modifiers. For example, in the test files that must not use UTF or -Unicode property support, this line appears: -.sp - < forbid 8W -.sp -This locks out the /8 and /W modifiers. An immediate error is given if they are -subsequently encountered. If the character string contains < but not >, all the -multi-character modifiers that begin with < are locked out. Otherwise, such -modifiers must be explicitly listed, for example: -.sp - < forbid <JS><cr> -.sp -There must be a single space between < and "forbid" for this feature to be -recognised. If there is not, the line is interpreted either as a request to -re-load a pre-compiled pattern (see "SAVING AND RELOADING COMPILED PATTERNS" -below) or, if there is a another < character, as a pattern that uses < as its -delimiter. -. -. -.SH "DATA LINES" -.rs -.sp -Before each data line is passed to \fBpcre[16|32]_exec()\fP, leading and trailing -white space is removed, and it is then scanned for \e escapes. Some of these -are pretty esoteric features, intended for checking out some of the more -complicated features of PCRE. If you are just testing "ordinary" regular -expressions, you probably don't need any of these. The following escapes are -recognized: -.sp - \ea alarm (BEL, \ex07) - \eb backspace (\ex08) - \ee escape (\ex27) - \ef form feed (\ex0c) - \en newline (\ex0a) -.\" JOIN - \eqdd set the PCRE_MATCH_LIMIT limit to dd - (any number of digits) - \er carriage return (\ex0d) - \et tab (\ex09) - \ev vertical tab (\ex0b) - \ennn octal character (up to 3 octal digits); always - a byte unless > 255 in UTF-8 or 16-bit or 32-bit mode - \eo{dd...} octal character (any number of octal digits} - \exhh hexadecimal byte (up to 2 hex digits) - \ex{hh...} hexadecimal character (any number of hex digits) -.\" JOIN - \eA pass the PCRE_ANCHORED option to \fBpcre[16|32]_exec()\fP - or \fBpcre[16|32]_dfa_exec()\fP -.\" JOIN - \eB pass the PCRE_NOTBOL option to \fBpcre[16|32]_exec()\fP - or \fBpcre[16|32]_dfa_exec()\fP -.\" JOIN - \eCdd call pcre[16|32]_copy_substring() for substring dd - after a successful match (number less than 32) -.\" JOIN - \eCname call pcre[16|32]_copy_named_substring() for substring - "name" after a successful match (name termin- - ated by next non alphanumeric character) -.\" JOIN - \eC+ show the current captured substrings at callout - time - \eC- do not supply a callout function -.\" JOIN - \eC!n return 1 instead of 0 when callout number n is - reached -.\" JOIN - \eC!n!m return 1 instead of 0 when callout number n is - reached for the nth time -.\" JOIN - \eC*n pass the number n (may be negative) as callout - data; this is used as the callout return value - \eD use the \fBpcre[16|32]_dfa_exec()\fP match function - \eF only shortest match for \fBpcre[16|32]_dfa_exec()\fP -.\" JOIN - \eGdd call pcre[16|32]_get_substring() for substring dd - after a successful match (number less than 32) -.\" JOIN - \eGname call pcre[16|32]_get_named_substring() for substring - "name" after a successful match (name termin- - ated by next non-alphanumeric character) -.\" JOIN - \eJdd set up a JIT stack of dd kilobytes maximum (any - number of digits) -.\" JOIN - \eL call pcre[16|32]_get_substringlist() after a - successful match -.\" JOIN - \eM discover the minimum MATCH_LIMIT and - MATCH_LIMIT_RECURSION settings -.\" JOIN - \eN pass the PCRE_NOTEMPTY option to \fBpcre[16|32]_exec()\fP - or \fBpcre[16|32]_dfa_exec()\fP; if used twice, pass the - PCRE_NOTEMPTY_ATSTART option -.\" JOIN - \eOdd set the size of the output vector passed to - \fBpcre[16|32]_exec()\fP to dd (any number of digits) -.\" JOIN - \eP pass the PCRE_PARTIAL_SOFT option to \fBpcre[16|32]_exec()\fP - or \fBpcre[16|32]_dfa_exec()\fP; if used twice, pass the - PCRE_PARTIAL_HARD option -.\" JOIN - \eQdd set the PCRE_MATCH_LIMIT_RECURSION limit to dd - (any number of digits) - \eR pass the PCRE_DFA_RESTART option to \fBpcre[16|32]_dfa_exec()\fP - \eS output details of memory get/free calls during matching -.\" JOIN - \eY pass the PCRE_NO_START_OPTIMIZE option to \fBpcre[16|32]_exec()\fP - or \fBpcre[16|32]_dfa_exec()\fP -.\" JOIN - \eZ pass the PCRE_NOTEOL option to \fBpcre[16|32]_exec()\fP - or \fBpcre[16|32]_dfa_exec()\fP -.\" JOIN - \e? pass the PCRE_NO_UTF[8|16|32]_CHECK option to - \fBpcre[16|32]_exec()\fP or \fBpcre[16|32]_dfa_exec()\fP -.\" JOIN - \e>dd start the match at offset dd (optional "-"; then - any number of digits); this sets the \fIstartoffset\fP - argument for \fBpcre[16|32]_exec()\fP or \fBpcre[16|32]_dfa_exec()\fP -.\" JOIN - \e<cr> pass the PCRE_NEWLINE_CR option to \fBpcre[16|32]_exec()\fP - or \fBpcre[16|32]_dfa_exec()\fP -.\" JOIN - \e<lf> pass the PCRE_NEWLINE_LF option to \fBpcre[16|32]_exec()\fP - or \fBpcre[16|32]_dfa_exec()\fP -.\" JOIN - \e<crlf> pass the PCRE_NEWLINE_CRLF option to \fBpcre[16|32]_exec()\fP - or \fBpcre[16|32]_dfa_exec()\fP -.\" JOIN - \e<anycrlf> pass the PCRE_NEWLINE_ANYCRLF option to \fBpcre[16|32]_exec()\fP - or \fBpcre[16|32]_dfa_exec()\fP -.\" JOIN - \e<any> pass the PCRE_NEWLINE_ANY option to \fBpcre[16|32]_exec()\fP - or \fBpcre[16|32]_dfa_exec()\fP -.sp -The use of \ex{hh...} is not dependent on the use of the \fB/8\fP modifier on -the pattern. It is recognized always. There may be any number of hexadecimal -digits inside the braces; invalid values provoke error messages. -.P -Note that \exhh specifies one byte rather than one character in UTF-8 mode; -this makes it possible to construct invalid UTF-8 sequences for testing -purposes. On the other hand, \ex{hh} is interpreted as a UTF-8 character in -UTF-8 mode, generating more than one byte if the value is greater than 127. -When testing the 8-bit library not in UTF-8 mode, \ex{hh} generates one byte -for values less than 256, and causes an error for greater values. -.P -In UTF-16 mode, all 4-digit \ex{hhhh} values are accepted. This makes it -possible to construct invalid UTF-16 sequences for testing purposes. -.P -In UTF-32 mode, all 4- to 8-digit \ex{...} values are accepted. This makes it -possible to construct invalid UTF-32 sequences for testing purposes. -.P -The escapes that specify line ending sequences are literal strings, exactly as -shown. No more than one newline setting should be present in any data line. -.P -A backslash followed by anything else just escapes the anything else. If -the very last character is a backslash, it is ignored. This gives a way of -passing an empty line as data, since a real empty line terminates the data -input. -.P -The \fB\eJ\fP escape provides a way of setting the maximum stack size that is -used by the just-in-time optimization code. It is ignored if JIT optimization -is not being used. Providing a stack that is larger than the default 32K is -necessary only for very complicated patterns. -.P -If \eM is present, \fBpcretest\fP calls \fBpcre[16|32]_exec()\fP several times, -with different values in the \fImatch_limit\fP and \fImatch_limit_recursion\fP -fields of the \fBpcre[16|32]_extra\fP data structure, until it finds the minimum -numbers for each parameter that allow \fBpcre[16|32]_exec()\fP to complete without -error. Because this is testing a specific feature of the normal interpretive -\fBpcre[16|32]_exec()\fP execution, the use of any JIT optimization that might -have been set up by the \fB/S+\fP qualifier of \fB-s+\fP option is disabled. -.P -The \fImatch_limit\fP number is a measure of the amount of backtracking -that takes place, and checking it out can be instructive. For most simple -matches, the number is quite small, but for patterns with very large numbers of -matching possibilities, it can become large very quickly with increasing length -of subject string. The \fImatch_limit_recursion\fP number is a measure of how -much stack (or, if PCRE is compiled with NO_RECURSE, how much heap) memory is -needed to complete the match attempt. -.P -When \eO is used, the value specified may be higher or lower than the size set -by the \fB-O\fP command line option (or defaulted to 45); \eO applies only to -the call of \fBpcre[16|32]_exec()\fP for the line in which it appears. -.P -If the \fB/P\fP modifier was present on the pattern, causing the POSIX wrapper -API to be used, the only option-setting sequences that have any effect are \eB, -\eN, and \eZ, causing REG_NOTBOL, REG_NOTEMPTY, and REG_NOTEOL, respectively, -to be passed to \fBregexec()\fP. -. -. -.SH "THE ALTERNATIVE MATCHING FUNCTION" -.rs -.sp -By default, \fBpcretest\fP uses the standard PCRE matching function, -\fBpcre[16|32]_exec()\fP to match each data line. PCRE also supports an -alternative matching function, \fBpcre[16|32]_dfa_test()\fP, which operates in a -different way, and has some restrictions. The differences between the two -functions are described in the -.\" HREF -\fBpcrematching\fP -.\" -documentation. -.P -If a data line contains the \eD escape sequence, or if the command line -contains the \fB-dfa\fP option, the alternative matching function is used. -This function finds all possible matches at a given point. If, however, the \eF -escape sequence is present in the data line, it stops after the first match is -found. This is always the shortest possible match. -. -. -.SH "DEFAULT OUTPUT FROM PCRETEST" -.rs -.sp -This section describes the output when the normal matching function, -\fBpcre[16|32]_exec()\fP, is being used. -.P -When a match succeeds, \fBpcretest\fP outputs the list of captured substrings -that \fBpcre[16|32]_exec()\fP returns, starting with number 0 for the string that -matched the whole pattern. Otherwise, it outputs "No match" when the return is -PCRE_ERROR_NOMATCH, and "Partial match:" followed by the partially matching -substring when \fBpcre[16|32]_exec()\fP returns PCRE_ERROR_PARTIAL. (Note that -this is the entire substring that was inspected during the partial match; it -may include characters before the actual match start if a lookbehind assertion, -\eK, \eb, or \eB was involved.) For any other return, \fBpcretest\fP outputs -the PCRE negative error number and a short descriptive phrase. If the error is -a failed UTF string check, the offset of the start of the failing character and -the reason code are also output, provided that the size of the output vector is -at least two. Here is an example of an interactive \fBpcretest\fP run. -.sp - $ pcretest - PCRE version 8.13 2011-04-30 -.sp - re> /^abc(\ed+)/ - data> abc123 - 0: abc123 - 1: 123 - data> xyz - No match -.sp -Unset capturing substrings that are not followed by one that is set are not -returned by \fBpcre[16|32]_exec()\fP, and are not shown by \fBpcretest\fP. In the -following example, there are two capturing substrings, but when the first data -line is matched, the second, unset substring is not shown. An "internal" unset -substring is shown as "<unset>", as for the second data line. -.sp - re> /(a)|(b)/ - data> a - 0: a - 1: a - data> b - 0: b - 1: <unset> - 2: b -.sp -If the strings contain any non-printing characters, they are output as \exhh -escapes if the value is less than 256 and UTF mode is not set. Otherwise they -are output as \ex{hh...} escapes. See below for the definition of non-printing -characters. If the pattern has the \fB/+\fP modifier, the output for substring -0 is followed by the the rest of the subject string, identified by "0+" like -this: -.sp - re> /cat/+ - data> cataract - 0: cat - 0+ aract -.sp -If the pattern has the \fB/g\fP or \fB/G\fP modifier, the results of successive -matching attempts are output in sequence, like this: -.sp - re> /\eBi(\ew\ew)/g - data> Mississippi - 0: iss - 1: ss - 0: iss - 1: ss - 0: ipp - 1: pp -.sp -"No match" is output only if the first match attempt fails. Here is an example -of a failure message (the offset 4 that is specified by \e>4 is past the end of -the subject string): -.sp - re> /xyz/ - data> xyz\e>4 - Error -24 (bad offset value) -.P -If any of the sequences \fB\eC\fP, \fB\eG\fP, or \fB\eL\fP are present in a -data line that is successfully matched, the substrings extracted by the -convenience functions are output with C, G, or L after the string number -instead of a colon. This is in addition to the normal full list. The string -length (that is, the return from the extraction function) is given in -parentheses after each string for \fB\eC\fP and \fB\eG\fP. -.P -Note that whereas patterns can be continued over several lines (a plain ">" -prompt is used for continuations), data lines may not. However newlines can be -included in data by means of the \en escape (or \er, \er\en, etc., depending on -the newline sequence setting). -. -. -. -.SH "OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION" -.rs -.sp -When the alternative matching function, \fBpcre[16|32]_dfa_exec()\fP, is used (by -means of the \eD escape sequence or the \fB-dfa\fP command line option), the -output consists of a list of all the matches that start at the first point in -the subject where there is at least one match. For example: -.sp - re> /(tang|tangerine|tan)/ - data> yellow tangerine\eD - 0: tangerine - 1: tang - 2: tan -.sp -(Using the normal matching function on this data finds only "tang".) The -longest matching string is always given first (and numbered zero). After a -PCRE_ERROR_PARTIAL return, the output is "Partial match:", followed by the -partially matching substring. (Note that this is the entire substring that was -inspected during the partial match; it may include characters before the actual -match start if a lookbehind assertion, \eK, \eb, or \eB was involved.) -.P -If \fB/g\fP is present on the pattern, the search for further matches resumes -at the end of the longest match. For example: -.sp - re> /(tang|tangerine|tan)/g - data> yellow tangerine and tangy sultana\eD - 0: tangerine - 1: tang - 2: tan - 0: tang - 1: tan - 0: tan -.sp -Since the matching function does not support substring capture, the escape -sequences that are concerned with captured substrings are not relevant. -. -. -.SH "RESTARTING AFTER A PARTIAL MATCH" -.rs -.sp -When the alternative matching function has given the PCRE_ERROR_PARTIAL return, -indicating that the subject partially matched the pattern, you can restart the -match with additional subject data by means of the \eR escape sequence. For -example: -.sp - re> /^\ed?\ed(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\ed\ed$/ - data> 23ja\eP\eD - Partial match: 23ja - data> n05\eR\eD - 0: n05 -.sp -For further information about partial matching, see the -.\" HREF -\fBpcrepartial\fP -.\" -documentation. -. -. -.SH CALLOUTS -.rs -.sp -If the pattern contains any callout requests, \fBpcretest\fP's callout function -is called during matching. This works with both matching functions. By default, -the called function displays the callout number, the start and current -positions in the text at the callout time, and the next pattern item to be -tested. For example: -.sp - --->pqrabcdef - 0 ^ ^ \ed -.sp -This output indicates that callout number 0 occurred for a match attempt -starting at the fourth character of the subject string, when the pointer was at -the seventh character of the data, and when the next pattern item was \ed. Just -one circumflex is output if the start and current positions are the same. -.P -Callouts numbered 255 are assumed to be automatic callouts, inserted as a -result of the \fB/C\fP pattern modifier. In this case, instead of showing the -callout number, the offset in the pattern, preceded by a plus, is output. For -example: -.sp - re> /\ed?[A-E]\e*/C - data> E* - --->E* - +0 ^ \ed? - +3 ^ [A-E] - +8 ^^ \e* - +10 ^ ^ - 0: E* -.sp -If a pattern contains (*MARK) items, an additional line is output whenever -a change of latest mark is passed to the callout function. For example: -.sp - re> /a(*MARK:X)bc/C - data> abc - --->abc - +0 ^ a - +1 ^^ (*MARK:X) - +10 ^^ b - Latest Mark: X - +11 ^ ^ c - +12 ^ ^ - 0: abc -.sp -The mark changes between matching "a" and "b", but stays the same for the rest -of the match, so nothing more is output. If, as a result of backtracking, the -mark reverts to being unset, the text "<unset>" is output. -.P -The callout function in \fBpcretest\fP returns zero (carry on matching) by -default, but you can use a \eC item in a data line (as described above) to -change this and other parameters of the callout. -.P -Inserting callouts can be helpful when using \fBpcretest\fP to check -complicated regular expressions. For further information about callouts, see -the -.\" HREF -\fBpcrecallout\fP -.\" -documentation. -. -. -. -.SH "NON-PRINTING CHARACTERS" -.rs -.sp -When \fBpcretest\fP is outputting text in the compiled version of a pattern, -bytes other than 32-126 are always treated as non-printing characters are are -therefore shown as hex escapes. -.P -When \fBpcretest\fP is outputting text that is a matched part of a subject -string, it behaves in the same way, unless a different locale has been set for -the pattern (using the \fB/L\fP modifier). In this case, the \fBisprint()\fP -function to distinguish printing and non-printing characters. -. -. -. -.SH "SAVING AND RELOADING COMPILED PATTERNS" -.rs -.sp -The facilities described in this section are not available when the POSIX -interface to PCRE is being used, that is, when the \fB/P\fP pattern modifier is -specified. -.P -When the POSIX interface is not in use, you can cause \fBpcretest\fP to write a -compiled pattern to a file, by following the modifiers with > and a file name. -For example: -.sp - /pattern/im >/some/file -.sp -See the -.\" HREF -\fBpcreprecompile\fP -.\" -documentation for a discussion about saving and re-using compiled patterns. -Note that if the pattern was successfully studied with JIT optimization, the -JIT data cannot be saved. -.P -The data that is written is binary. The first eight bytes are the length of the -compiled pattern data followed by the length of the optional study data, each -written as four bytes in big-endian order (most significant byte first). If -there is no study data (either the pattern was not studied, or studying did not -return any data), the second length is zero. The lengths are followed by an -exact copy of the compiled pattern. If there is additional study data, this -(excluding any JIT data) follows immediately after the compiled pattern. After -writing the file, \fBpcretest\fP expects to read a new pattern. -.P -A saved pattern can be reloaded into \fBpcretest\fP by specifying < and a file -name instead of a pattern. There must be no space between < and the file name, -which must not contain a < character, as otherwise \fBpcretest\fP will -interpret the line as a pattern delimited by < characters. For example: -.sp - re> </some/file - Compiled pattern loaded from /some/file - No study data -.sp -If the pattern was previously studied with the JIT optimization, the JIT -information cannot be saved and restored, and so is lost. When the pattern has -been loaded, \fBpcretest\fP proceeds to read data lines in the usual way. -.P -You can copy a file written by \fBpcretest\fP to a different host and reload it -there, even if the new host has opposite endianness to the one on which the -pattern was compiled. For example, you can compile on an i86 machine and run on -a SPARC machine. When a pattern is reloaded on a host with different -endianness, the confirmation message is changed to: -.sp - Compiled pattern (byte-inverted) loaded from /some/file -.sp -The test suite contains some saved pre-compiled patterns with different -endianness. These are reloaded using "<!" instead of just "<". This suppresses -the "(byte-inverted)" text so that the output is the same on all hosts. It also -forces debugging output once the pattern has been reloaded. -.P -File names for saving and reloading can be absolute or relative, but note that -the shell facility of expanding a file name that starts with a tilde (~) is not -available. -.P -The ability to save and reload files in \fBpcretest\fP is intended for testing -and experimentation. It is not intended for production use because only a -single pattern can be written to a file. Furthermore, there is no facility for -supplying custom character tables for use with a reloaded pattern. If the -original pattern was compiled with custom tables, an attempt to match a subject -string using a reloaded pattern is likely to cause \fBpcretest\fP to crash. -Finally, if you attempt to load a file that is not in the correct format, the -result is undefined. -. -. -.SH "SEE ALSO" -.rs -.sp -\fBpcre\fP(3), \fBpcre16\fP(3), \fBpcre32\fP(3), \fBpcreapi\fP(3), -\fBpcrecallout\fP(3), -\fBpcrejit\fP, \fBpcrematching\fP(3), \fBpcrepartial\fP(d), -\fBpcrepattern\fP(3), \fBpcreprecompile\fP(3). -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 09 February 2014 -Copyright (c) 1997-2014 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/pcretest.txt b/plugins/Pcre16/docs/doc/pcretest.txt deleted file mode 100644 index 55de502443..0000000000 --- a/plugins/Pcre16/docs/doc/pcretest.txt +++ /dev/null @@ -1,1087 +0,0 @@ -PCRETEST(1) General Commands Manual PCRETEST(1) - - - -NAME - pcretest - a program for testing Perl-compatible regular expressions. - -SYNOPSIS - - pcretest [options] [input file [output file]] - - pcretest was written as a test program for the PCRE regular expression - library itself, but it can also be used for experimenting with regular - expressions. This document describes the features of the test program; - for details of the regular expressions themselves, see the pcrepattern - documentation. For details of the PCRE library function calls and their - options, see the pcreapi , pcre16 and pcre32 documentation. - - The input for pcretest is a sequence of regular expression patterns and - strings to be matched, as described below. The output shows the result - of each match. Options on the command line and the patterns control - PCRE options and exactly what is output. - - As PCRE has evolved, it has acquired many different features, and as a - result, pcretest now has rather a lot of obscure options for testing - every possible feature. Some of these options are specifically designed - for use in conjunction with the test script and data files that are - distributed as part of PCRE, and are unlikely to be of use otherwise. - They are all documented here, but without much justification. - - -INPUT DATA FORMAT - - Input to pcretest is processed line by line, either by calling the C - library's fgets() function, or via the libreadline library (see below). - In Unix-like environments, fgets() treats any bytes other than newline - as data characters. However, in some Windows environments character 26 - (hex 1A) causes an immediate end of file, and no further data is read. - For maximum portability, therefore, it is safest to use only ASCII - characters in pcretest input files. - - -PCRE's 8-BIT, 16-BIT AND 32-BIT LIBRARIES - - From release 8.30, two separate PCRE libraries can be built. The origi- - nal one supports 8-bit character strings, whereas the newer 16-bit - library supports character strings encoded in 16-bit units. From - release 8.32, a third library can be built, supporting character - strings encoded in 32-bit units. The pcretest program can be used to - test all three libraries. However, it is itself still an 8-bit program, - reading 8-bit input and writing 8-bit output. When testing the 16-bit - or 32-bit library, the patterns and data strings are converted to 16- - or 32-bit format before being passed to the PCRE library functions. - Results are converted to 8-bit for output. - - References to functions and structures of the form pcre[16|32]_xx below - mean "pcre_xx when using the 8-bit library, pcre16_xx when using the - 16-bit library, or pcre32_xx when using the 32-bit library". - - -COMMAND LINE OPTIONS - - -8 If both the 8-bit library has been built, this option causes - the 8-bit library to be used (which is the default); if the - 8-bit library has not been built, this option causes an - error. - - -16 If both the 8-bit or the 32-bit, and the 16-bit libraries - have been built, this option causes the 16-bit library to be - used. If only the 16-bit library has been built, this is the - default (so has no effect). If only the 8-bit or the 32-bit - library has been built, this option causes an error. - - -32 If both the 8-bit or the 16-bit, and the 32-bit libraries - have been built, this option causes the 32-bit library to be - used. If only the 32-bit library has been built, this is the - default (so has no effect). If only the 8-bit or the 16-bit - library has been built, this option causes an error. - - -b Behave as if each pattern has the /B (show byte code) modi- - fier; the internal form is output after compilation. - - -C Output the version number of the PCRE library, and all avail- - able information about the optional features that are - included, and then exit with zero exit code. All other - options are ignored. - - -C option Output information about a specific build-time option, then - exit. This functionality is intended for use in scripts such - as RunTest. The following options output the value and set - the exit code as indicated: - - ebcdic-nl the code for LF (= NL) in an EBCDIC environment: - 0x15 or 0x25 - 0 if used in an ASCII environment - exit code is always 0 - linksize the configured internal link size (2, 3, or 4) - exit code is set to the link size - newline the default newline setting: - CR, LF, CRLF, ANYCRLF, or ANY - exit code is always 0 - bsr the default setting for what \R matches: - ANYCRLF or ANY - exit code is always 0 - - The following options output 1 for true or 0 for false, and - set the exit code to the same value: - - ebcdic compiled for an EBCDIC environment - jit just-in-time support is available - pcre16 the 16-bit library was built - pcre32 the 32-bit library was built - pcre8 the 8-bit library was built - ucp Unicode property support is available - utf UTF-8 and/or UTF-16 and/or UTF-32 support - is available - - If an unknown option is given, an error message is output; - the exit code is 0. - - -d Behave as if each pattern has the /D (debug) modifier; the - internal form and information about the compiled pattern is - output after compilation; -d is equivalent to -b -i. - - -dfa Behave as if each data line contains the \D escape sequence; - this causes the alternative matching function, - pcre[16|32]_dfa_exec(), to be used instead of the standard - pcre[16|32]_exec() function (more detail is given below). - - -help Output a brief summary these options and then exit. - - -i Behave as if each pattern has the /I modifier; information - about the compiled pattern is given after compilation. - - -M Behave as if each data line contains the \M escape sequence; - this causes PCRE to discover the minimum MATCH_LIMIT and - MATCH_LIMIT_RECURSION settings by calling pcre[16|32]_exec() - repeatedly with different limits. - - -m Output the size of each compiled pattern after it has been - compiled. This is equivalent to adding /M to each regular - expression. The size is given in bytes for both libraries. - - -O Behave as if each pattern has the /O modifier, that is dis- - able auto-possessification for all patterns. - - -o osize Set the number of elements in the output vector that is used - when calling pcre[16|32]_exec() or pcre[16|32]_dfa_exec() to - be osize. The default value is 45, which is enough for 14 - capturing subexpressions for pcre[16|32]_exec() or 22 differ- - ent matches for pcre[16|32]_dfa_exec(). The vector size can - be changed for individual matching calls by including \O in - the data line (see below). - - -p Behave as if each pattern has the /P modifier; the POSIX - wrapper API is used to call PCRE. None of the other options - has any effect when -p is set. This option can be used only - with the 8-bit library. - - -q Do not output the version number of pcretest at the start of - execution. - - -S size On Unix-like systems, set the size of the run-time stack to - size megabytes. - - -s or -s+ Behave as if each pattern has the /S modifier; in other - words, force each pattern to be studied. If -s+ is used, all - the JIT compile options are passed to pcre[16|32]_study(), - causing just-in-time optimization to be set up if it is - available, for both full and partial matching. Specific JIT - compile options can be selected by following -s+ with a digit - in the range 1 to 7, which selects the JIT compile modes as - follows: - - 1 normal match only - 2 soft partial match only - 3 normal match and soft partial match - 4 hard partial match only - 6 soft and hard partial match - 7 all three modes (default) - - If -s++ is used instead of -s+ (with or without a following - digit), the text "(JIT)" is added to the first output line - after a match or no match when JIT-compiled code was actually - used. - - Note that there are pattern options that can override -s, - either specifying no studying at all, or suppressing JIT com- - pilation. - - If the /I or /D option is present on a pattern (requesting - output about the compiled pattern), information about the - result of studying is not included when studying is caused - only by -s and neither -i nor -d is present on the command - line. This behaviour means that the output from tests that - are run with and without -s should be identical, except when - options that output information about the actual running of a - match are set. - - The -M, -t, and -tm options, which give information about - resources used, are likely to produce different output with - and without -s. Output may also differ if the /C option is - present on an individual pattern. This uses callouts to trace - the the matching process, and this may be different between - studied and non-studied patterns. If the pattern contains - (*MARK) items there may also be differences, for the same - reason. The -s command line option can be overridden for spe- - cific patterns that should never be studied (see the /S pat- - tern modifier below). - - -t Run each compile, study, and match many times with a timer, - and output the resulting times per compile, study, or match - (in milliseconds). Do not set -m with -t, because you will - then get the size output a zillion times, and the timing will - be distorted. You can control the number of iterations that - are used for timing by following -t with a number (as a sepa- - rate item on the command line). For example, "-t 1000" iter- - ates 1000 times. The default is to iterate 500000 times. - - -tm This is like -t except that it times only the matching phase, - not the compile or study phases. - - -T -TM These behave like -t and -tm, but in addition, at the end of - a run, the total times for all compiles, studies, and matches - are output. - - -DESCRIPTION - - If pcretest is given two filename arguments, it reads from the first - and writes to the second. If it is given only one filename argument, it - reads from that file and writes to stdout. Otherwise, it reads from - stdin and writes to stdout, and prompts for each line of input, using - "re>" to prompt for regular expressions, and "data>" to prompt for data - lines. - - When pcretest is built, a configuration option can specify that it - should be linked with the libreadline library. When this is done, if - the input is from a terminal, it is read using the readline() function. - This provides line-editing and history facilities. The output from the - -help option states whether or not readline() will be used. - - The program handles any number of sets of input on a single input file. - Each set starts with a regular expression, and continues with any num- - ber of data lines to be matched against that pattern. - - Each data line is matched separately and independently. If you want to - do multi-line matches, you have to use the \n escape sequence (or \r or - \r\n, etc., depending on the newline setting) in a single line of input - to encode the newline sequences. There is no limit on the length of - data lines; the input buffer is automatically extended if it is too - small. - - An empty line signals the end of the data lines, at which point a new - regular expression is read. The regular expressions are given enclosed - in any non-alphanumeric delimiters other than backslash, for example: - - /(a|bc)x+yz/ - - White space before the initial delimiter is ignored. A regular expres- - sion may be continued over several input lines, in which case the new- - line characters are included within it. It is possible to include the - delimiter within the pattern by escaping it, for example - - /abc\/def/ - - If you do so, the escape and the delimiter form part of the pattern, - but since delimiters are always non-alphanumeric, this does not affect - its interpretation. If the terminating delimiter is immediately fol- - lowed by a backslash, for example, - - /abc/\ - - then a backslash is added to the end of the pattern. This is done to - provide a way of testing the error condition that arises if a pattern - finishes with a backslash, because - - /abc\/ - - is interpreted as the first line of a pattern that starts with "abc/", - causing pcretest to read the next line as a continuation of the regular - expression. - - -PATTERN MODIFIERS - - A pattern may be followed by any number of modifiers, which are mostly - single characters, though some of these can be qualified by further - characters. Following Perl usage, these are referred to below as, for - example, "the /i modifier", even though the delimiter of the pattern - need not always be a slash, and no slash is used when writing modi- - fiers. White space may appear between the final pattern delimiter and - the first modifier, and between the modifiers themselves. For refer- - ence, here is a complete list of modifiers. They fall into several - groups that are described in detail in the following sections. - - /8 set UTF mode - /9 set PCRE_NEVER_UTF (locks out UTF mode) - /? disable UTF validity check - /+ show remainder of subject after match - /= show all captures (not just those that are set) - - /A set PCRE_ANCHORED - /B show compiled code - /C set PCRE_AUTO_CALLOUT - /D same as /B plus /I - /E set PCRE_DOLLAR_ENDONLY - /F flip byte order in compiled pattern - /f set PCRE_FIRSTLINE - /G find all matches (shorten string) - /g find all matches (use startoffset) - /I show information about pattern - /i set PCRE_CASELESS - /J set PCRE_DUPNAMES - /K show backtracking control names - /L set locale - /M show compiled memory size - /m set PCRE_MULTILINE - /N set PCRE_NO_AUTO_CAPTURE - /O set PCRE_NO_AUTO_POSSESS - /P use the POSIX wrapper - /Q test external stack check function - /S study the pattern after compilation - /s set PCRE_DOTALL - /T select character tables - /U set PCRE_UNGREEDY - /W set PCRE_UCP - /X set PCRE_EXTRA - /x set PCRE_EXTENDED - /Y set PCRE_NO_START_OPTIMIZE - /Z don't show lengths in /B output - - /<any> set PCRE_NEWLINE_ANY - /<anycrlf> set PCRE_NEWLINE_ANYCRLF - /<cr> set PCRE_NEWLINE_CR - /<crlf> set PCRE_NEWLINE_CRLF - /<lf> set PCRE_NEWLINE_LF - /<bsr_anycrlf> set PCRE_BSR_ANYCRLF - /<bsr_unicode> set PCRE_BSR_UNICODE - /<JS> set PCRE_JAVASCRIPT_COMPAT - - - Perl-compatible modifiers - - The /i, /m, /s, and /x modifiers set the PCRE_CASELESS, PCRE_MULTILINE, - PCRE_DOTALL, or PCRE_EXTENDED options, respectively, when - pcre[16|32]_compile() is called. These four modifier letters have the - same effect as they do in Perl. For example: - - /caseless/i - - - Modifiers for other PCRE options - - The following table shows additional modifiers for setting PCRE com- - pile-time options that do not correspond to anything in Perl: - - /8 PCRE_UTF8 ) when using the 8-bit - /? PCRE_NO_UTF8_CHECK ) library - - /8 PCRE_UTF16 ) when using the 16-bit - /? PCRE_NO_UTF16_CHECK ) library - - /8 PCRE_UTF32 ) when using the 32-bit - /? PCRE_NO_UTF32_CHECK ) library - - /9 PCRE_NEVER_UTF - /A PCRE_ANCHORED - /C PCRE_AUTO_CALLOUT - /E PCRE_DOLLAR_ENDONLY - /f PCRE_FIRSTLINE - /J PCRE_DUPNAMES - /N PCRE_NO_AUTO_CAPTURE - /O PCRE_NO_AUTO_POSSESS - /U PCRE_UNGREEDY - /W PCRE_UCP - /X PCRE_EXTRA - /Y PCRE_NO_START_OPTIMIZE - /<any> PCRE_NEWLINE_ANY - /<anycrlf> PCRE_NEWLINE_ANYCRLF - /<cr> PCRE_NEWLINE_CR - /<crlf> PCRE_NEWLINE_CRLF - /<lf> PCRE_NEWLINE_LF - /<bsr_anycrlf> PCRE_BSR_ANYCRLF - /<bsr_unicode> PCRE_BSR_UNICODE - /<JS> PCRE_JAVASCRIPT_COMPAT - - The modifiers that are enclosed in angle brackets are literal strings - as shown, including the angle brackets, but the letters within can be - in either case. This example sets multiline matching with CRLF as the - line ending sequence: - - /^abc/m<CRLF> - - As well as turning on the PCRE_UTF8/16/32 option, the /8 modifier - causes all non-printing characters in output strings to be printed - using the \x{hh...} notation. Otherwise, those less than 0x100 are out- - put in hex without the curly brackets. - - Full details of the PCRE options are given in the pcreapi documenta- - tion. - - Finding all matches in a string - - Searching for all possible matches within each subject string can be - requested by the /g or /G modifier. After finding a match, PCRE is - called again to search the remainder of the subject string. The differ- - ence between /g and /G is that the former uses the startoffset argument - to pcre[16|32]_exec() to start searching at a new point within the - entire string (which is in effect what Perl does), whereas the latter - passes over a shortened substring. This makes a difference to the - matching process if the pattern begins with a lookbehind assertion - (including \b or \B). - - If any call to pcre[16|32]_exec() in a /g or /G sequence matches an - empty string, the next call is done with the PCRE_NOTEMPTY_ATSTART and - PCRE_ANCHORED flags set in order to search for another, non-empty, - match at the same point. If this second match fails, the start offset - is advanced, and the normal match is retried. This imitates the way - Perl handles such cases when using the /g modifier or the split() func- - tion. Normally, the start offset is advanced by one character, but if - the newline convention recognizes CRLF as a newline, and the current - character is CR followed by LF, an advance of two is used. - - Other modifiers - - There are yet more modifiers for controlling the way pcretest operates. - - The /+ modifier requests that as well as outputting the substring that - matched the entire pattern, pcretest should in addition output the - remainder of the subject string. This is useful for tests where the - subject contains multiple copies of the same substring. If the + modi- - fier appears twice, the same action is taken for captured substrings. - In each case the remainder is output on the following line with a plus - character following the capture number. Note that this modifier must - not immediately follow the /S modifier because /S+ and /S++ have other - meanings. - - The /= modifier requests that the values of all potential captured - parentheses be output after a match. By default, only those up to the - highest one actually used in the match are output (corresponding to the - return code from pcre[16|32]_exec()). Values in the offsets vector cor- - responding to higher numbers should be set to -1, and these are output - as "<unset>". This modifier gives a way of checking that this is hap- - pening. - - The /B modifier is a debugging feature. It requests that pcretest out- - put a representation of the compiled code after compilation. Normally - this information contains length and offset values; however, if /Z is - also present, this data is replaced by spaces. This is a special fea- - ture for use in the automatic test scripts; it ensures that the same - output is generated for different internal link sizes. - - The /D modifier is a PCRE debugging feature, and is equivalent to /BI, - that is, both the /B and the /I modifiers. - - The /F modifier causes pcretest to flip the byte order of the 2-byte - and 4-byte fields in the compiled pattern. This facility is for testing - the feature in PCRE that allows it to execute patterns that were com- - piled on a host with a different endianness. This feature is not avail- - able when the POSIX interface to PCRE is being used, that is, when the - /P pattern modifier is specified. See also the section about saving and - reloading compiled patterns below. - - The /I modifier requests that pcretest output information about the - compiled pattern (whether it is anchored, has a fixed first character, - and so on). It does this by calling pcre[16|32]_fullinfo() after com- - piling a pattern. If the pattern is studied, the results of that are - also output. In this output, the word "char" means a non-UTF character, - that is, the value of a single data item (8-bit, 16-bit, or 32-bit, - depending on the library that is being tested). - - The /K modifier requests pcretest to show names from backtracking con- - trol verbs that are returned from calls to pcre[16|32]_exec(). It - causes pcretest to create a pcre[16|32]_extra block if one has not - already been created by a call to pcre[16|32]_study(), and to set the - PCRE_EXTRA_MARK flag and the mark field within it, every time that - pcre[16|32]_exec() is called. If the variable that the mark field - points to is non-NULL for a match, non-match, or partial match, - pcretest prints the string to which it points. For a match, this is - shown on a line by itself, tagged with "MK:". For a non-match it is - added to the message. - - The /L modifier must be followed directly by the name of a locale, for - example, - - /pattern/Lfr_FR - - For this reason, it must be the last modifier. The given locale is set, - pcre[16|32]_maketables() is called to build a set of character tables - for the locale, and this is then passed to pcre[16|32]_compile() when - compiling the regular expression. Without an /L (or /T) modifier, NULL - is passed as the tables pointer; that is, /L applies only to the - expression on which it appears. - - The /M modifier causes the size in bytes of the memory block used to - hold the compiled pattern to be output. This does not include the size - of the pcre[16|32] block; it is just the actual compiled data. If the - pattern is successfully studied with the PCRE_STUDY_JIT_COMPILE option, - the size of the JIT compiled code is also output. - - The /Q modifier is used to test the use of pcre_stack_guard. It must be - followed by '0' or '1', specifying the return code to be given from an - external function that is passed to PCRE and used for stack checking - during compilation (see the pcreapi documentation for details). - - The /S modifier causes pcre[16|32]_study() to be called after the - expression has been compiled, and the results used when the expression - is matched. There are a number of qualifying characters that may follow - /S. They may appear in any order. - - If /S is followed by an exclamation mark, pcre[16|32]_study() is called - with the PCRE_STUDY_EXTRA_NEEDED option, causing it always to return a - pcre_extra block, even when studying discovers no useful information. - - If /S is followed by a second S character, it suppresses studying, even - if it was requested externally by the -s command line option. This - makes it possible to specify that certain patterns are always studied, - and others are never studied, independently of -s. This feature is used - in the test files in a few cases where the output is different when the - pattern is studied. - - If the /S modifier is followed by a + character, the call to - pcre[16|32]_study() is made with all the JIT study options, requesting - just-in-time optimization support if it is available, for both normal - and partial matching. If you want to restrict the JIT compiling modes, - you can follow /S+ with a digit in the range 1 to 7: - - 1 normal match only - 2 soft partial match only - 3 normal match and soft partial match - 4 hard partial match only - 6 soft and hard partial match - 7 all three modes (default) - - If /S++ is used instead of /S+ (with or without a following digit), the - text "(JIT)" is added to the first output line after a match or no - match when JIT-compiled code was actually used. - - Note that there is also an independent /+ modifier; it must not be - given immediately after /S or /S+ because this will be misinterpreted. - - If JIT studying is successful, the compiled JIT code will automatically - be used when pcre[16|32]_exec() is run, except when incompatible run- - time options are specified. For more details, see the pcrejit documen- - tation. See also the \J escape sequence below for a way of setting the - size of the JIT stack. - - Finally, if /S is followed by a minus character, JIT compilation is - suppressed, even if it was requested externally by the -s command line - option. This makes it possible to specify that JIT is never to be used - for certain patterns. - - The /T modifier must be followed by a single digit. It causes a spe- - cific set of built-in character tables to be passed to pcre[16|32]_com- - pile(). It is used in the standard PCRE tests to check behaviour with - different character tables. The digit specifies the tables as follows: - - 0 the default ASCII tables, as distributed in - pcre_chartables.c.dist - 1 a set of tables defining ISO 8859 characters - - In table 1, some characters whose codes are greater than 128 are iden- - tified as letters, digits, spaces, etc. - - Using the POSIX wrapper API - - The /P modifier causes pcretest to call PCRE via the POSIX wrapper API - rather than its native API. This supports only the 8-bit library. When - /P is set, the following modifiers set options for the regcomp() func- - tion: - - /i REG_ICASE - /m REG_NEWLINE - /N REG_NOSUB - /s REG_DOTALL ) - /U REG_UNGREEDY ) These options are not part of - /W REG_UCP ) the POSIX standard - /8 REG_UTF8 ) - - The /+ modifier works as described above. All other modifiers are - ignored. - - Locking out certain modifiers - - PCRE can be compiled with or without support for certain features such - as UTF-8/16/32 or Unicode properties. Accordingly, the standard tests - are split up into a number of different files that are selected for - running depending on which features are available. When updating the - tests, it is all too easy to put a new test into the wrong file by mis- - take; for example, to put a test that requires UTF support into a file - that is used when it is not available. To help detect such mistakes as - early as possible, there is a facility for locking out specific modi- - fiers. If an input line for pcretest starts with the string "< forbid " - the following sequence of characters is taken as a list of forbidden - modifiers. For example, in the test files that must not use UTF or Uni- - code property support, this line appears: - - < forbid 8W - - This locks out the /8 and /W modifiers. An immediate error is given if - they are subsequently encountered. If the character string contains < - but not >, all the multi-character modifiers that begin with < are - locked out. Otherwise, such modifiers must be explicitly listed, for - example: - - < forbid <JS><cr> - - There must be a single space between < and "forbid" for this feature to - be recognised. If there is not, the line is interpreted either as a - request to re-load a pre-compiled pattern (see "SAVING AND RELOADING - COMPILED PATTERNS" below) or, if there is a another < character, as a - pattern that uses < as its delimiter. - - -DATA LINES - - Before each data line is passed to pcre[16|32]_exec(), leading and - trailing white space is removed, and it is then scanned for \ escapes. - Some of these are pretty esoteric features, intended for checking out - some of the more complicated features of PCRE. If you are just testing - "ordinary" regular expressions, you probably don't need any of these. - The following escapes are recognized: - - \a alarm (BEL, \x07) - \b backspace (\x08) - \e escape (\x27) - \f form feed (\x0c) - \n newline (\x0a) - \qdd set the PCRE_MATCH_LIMIT limit to dd - (any number of digits) - \r carriage return (\x0d) - \t tab (\x09) - \v vertical tab (\x0b) - \nnn octal character (up to 3 octal digits); always - a byte unless > 255 in UTF-8 or 16-bit or 32-bit mode - \o{dd...} octal character (any number of octal digits} - \xhh hexadecimal byte (up to 2 hex digits) - \x{hh...} hexadecimal character (any number of hex digits) - \A pass the PCRE_ANCHORED option to pcre[16|32]_exec() - or pcre[16|32]_dfa_exec() - \B pass the PCRE_NOTBOL option to pcre[16|32]_exec() - or pcre[16|32]_dfa_exec() - \Cdd call pcre[16|32]_copy_substring() for substring dd - after a successful match (number less than 32) - \Cname call pcre[16|32]_copy_named_substring() for substring - "name" after a successful match (name termin- - ated by next non alphanumeric character) - \C+ show the current captured substrings at callout - time - \C- do not supply a callout function - \C!n return 1 instead of 0 when callout number n is - reached - \C!n!m return 1 instead of 0 when callout number n is - reached for the nth time - \C*n pass the number n (may be negative) as callout - data; this is used as the callout return value - \D use the pcre[16|32]_dfa_exec() match function - \F only shortest match for pcre[16|32]_dfa_exec() - \Gdd call pcre[16|32]_get_substring() for substring dd - after a successful match (number less than 32) - \Gname call pcre[16|32]_get_named_substring() for substring - "name" after a successful match (name termin- - ated by next non-alphanumeric character) - \Jdd set up a JIT stack of dd kilobytes maximum (any - number of digits) - \L call pcre[16|32]_get_substringlist() after a - successful match - \M discover the minimum MATCH_LIMIT and - MATCH_LIMIT_RECURSION settings - \N pass the PCRE_NOTEMPTY option to pcre[16|32]_exec() - or pcre[16|32]_dfa_exec(); if used twice, pass the - PCRE_NOTEMPTY_ATSTART option - \Odd set the size of the output vector passed to - pcre[16|32]_exec() to dd (any number of digits) - \P pass the PCRE_PARTIAL_SOFT option to pcre[16|32]_exec() - or pcre[16|32]_dfa_exec(); if used twice, pass the - PCRE_PARTIAL_HARD option - \Qdd set the PCRE_MATCH_LIMIT_RECURSION limit to dd - (any number of digits) - \R pass the PCRE_DFA_RESTART option to pcre[16|32]_dfa_exec() - \S output details of memory get/free calls during matching - \Y pass the PCRE_NO_START_OPTIMIZE option to - pcre[16|32]_exec() - or pcre[16|32]_dfa_exec() - \Z pass the PCRE_NOTEOL option to pcre[16|32]_exec() - or pcre[16|32]_dfa_exec() - \? pass the PCRE_NO_UTF[8|16|32]_CHECK option to - pcre[16|32]_exec() or pcre[16|32]_dfa_exec() - \>dd start the match at offset dd (optional "-"; then - any number of digits); this sets the startoffset - argument for pcre[16|32]_exec() or - pcre[16|32]_dfa_exec() - \<cr> pass the PCRE_NEWLINE_CR option to pcre[16|32]_exec() - or pcre[16|32]_dfa_exec() - \<lf> pass the PCRE_NEWLINE_LF option to pcre[16|32]_exec() - or pcre[16|32]_dfa_exec() - \<crlf> pass the PCRE_NEWLINE_CRLF option to pcre[16|32]_exec() - or pcre[16|32]_dfa_exec() - \<anycrlf> pass the PCRE_NEWLINE_ANYCRLF option to pcre[16|32]_exec() - or pcre[16|32]_dfa_exec() - \<any> pass the PCRE_NEWLINE_ANY option to pcre[16|32]_exec() - or pcre[16|32]_dfa_exec() - - The use of \x{hh...} is not dependent on the use of the /8 modifier on - the pattern. It is recognized always. There may be any number of hexa- - decimal digits inside the braces; invalid values provoke error mes- - sages. - - Note that \xhh specifies one byte rather than one character in UTF-8 - mode; this makes it possible to construct invalid UTF-8 sequences for - testing purposes. On the other hand, \x{hh} is interpreted as a UTF-8 - character in UTF-8 mode, generating more than one byte if the value is - greater than 127. When testing the 8-bit library not in UTF-8 mode, - \x{hh} generates one byte for values less than 256, and causes an error - for greater values. - - In UTF-16 mode, all 4-digit \x{hhhh} values are accepted. This makes it - possible to construct invalid UTF-16 sequences for testing purposes. - - In UTF-32 mode, all 4- to 8-digit \x{...} values are accepted. This - makes it possible to construct invalid UTF-32 sequences for testing - purposes. - - The escapes that specify line ending sequences are literal strings, - exactly as shown. No more than one newline setting should be present in - any data line. - - A backslash followed by anything else just escapes the anything else. - If the very last character is a backslash, it is ignored. This gives a - way of passing an empty line as data, since a real empty line termi- - nates the data input. - - The \J escape provides a way of setting the maximum stack size that is - used by the just-in-time optimization code. It is ignored if JIT opti- - mization is not being used. Providing a stack that is larger than the - default 32K is necessary only for very complicated patterns. - - If \M is present, pcretest calls pcre[16|32]_exec() several times, with - different values in the match_limit and match_limit_recursion fields of - the pcre[16|32]_extra data structure, until it finds the minimum num- - bers for each parameter that allow pcre[16|32]_exec() to complete with- - out error. Because this is testing a specific feature of the normal - interpretive pcre[16|32]_exec() execution, the use of any JIT optimiza- - tion that might have been set up by the /S+ qualifier of -s+ option is - disabled. - - The match_limit number is a measure of the amount of backtracking that - takes place, and checking it out can be instructive. For most simple - matches, the number is quite small, but for patterns with very large - numbers of matching possibilities, it can become large very quickly - with increasing length of subject string. The match_limit_recursion - number is a measure of how much stack (or, if PCRE is compiled with - NO_RECURSE, how much heap) memory is needed to complete the match - attempt. - - When \O is used, the value specified may be higher or lower than the - size set by the -O command line option (or defaulted to 45); \O applies - only to the call of pcre[16|32]_exec() for the line in which it - appears. - - If the /P modifier was present on the pattern, causing the POSIX wrap- - per API to be used, the only option-setting sequences that have any - effect are \B, \N, and \Z, causing REG_NOTBOL, REG_NOTEMPTY, and - REG_NOTEOL, respectively, to be passed to regexec(). - - -THE ALTERNATIVE MATCHING FUNCTION - - By default, pcretest uses the standard PCRE matching function, - pcre[16|32]_exec() to match each data line. PCRE also supports an - alternative matching function, pcre[16|32]_dfa_test(), which operates - in a different way, and has some restrictions. The differences between - the two functions are described in the pcrematching documentation. - - If a data line contains the \D escape sequence, or if the command line - contains the -dfa option, the alternative matching function is used. - This function finds all possible matches at a given point. If, however, - the \F escape sequence is present in the data line, it stops after the - first match is found. This is always the shortest possible match. - - -DEFAULT OUTPUT FROM PCRETEST - - This section describes the output when the normal matching function, - pcre[16|32]_exec(), is being used. - - When a match succeeds, pcretest outputs the list of captured substrings - that pcre[16|32]_exec() returns, starting with number 0 for the string - that matched the whole pattern. Otherwise, it outputs "No match" when - the return is PCRE_ERROR_NOMATCH, and "Partial match:" followed by the - partially matching substring when pcre[16|32]_exec() returns - PCRE_ERROR_PARTIAL. (Note that this is the entire substring that was - inspected during the partial match; it may include characters before - the actual match start if a lookbehind assertion, \K, \b, or \B was - involved.) For any other return, pcretest outputs the PCRE negative - error number and a short descriptive phrase. If the error is a failed - UTF string check, the offset of the start of the failing character and - the reason code are also output, provided that the size of the output - vector is at least two. Here is an example of an interactive pcretest - run. - - $ pcretest - PCRE version 8.13 2011-04-30 - - re> /^abc(\d+)/ - data> abc123 - 0: abc123 - 1: 123 - data> xyz - No match - - Unset capturing substrings that are not followed by one that is set are - not returned by pcre[16|32]_exec(), and are not shown by pcretest. In - the following example, there are two capturing substrings, but when the - first data line is matched, the second, unset substring is not shown. - An "internal" unset substring is shown as "<unset>", as for the second - data line. - - re> /(a)|(b)/ - data> a - 0: a - 1: a - data> b - 0: b - 1: <unset> - 2: b - - If the strings contain any non-printing characters, they are output as - \xhh escapes if the value is less than 256 and UTF mode is not set. - Otherwise they are output as \x{hh...} escapes. See below for the defi- - nition of non-printing characters. If the pattern has the /+ modifier, - the output for substring 0 is followed by the the rest of the subject - string, identified by "0+" like this: - - re> /cat/+ - data> cataract - 0: cat - 0+ aract - - If the pattern has the /g or /G modifier, the results of successive - matching attempts are output in sequence, like this: - - re> /\Bi(\w\w)/g - data> Mississippi - 0: iss - 1: ss - 0: iss - 1: ss - 0: ipp - 1: pp - - "No match" is output only if the first match attempt fails. Here is an - example of a failure message (the offset 4 that is specified by \>4 is - past the end of the subject string): - - re> /xyz/ - data> xyz\>4 - Error -24 (bad offset value) - - If any of the sequences \C, \G, or \L are present in a data line that - is successfully matched, the substrings extracted by the convenience - functions are output with C, G, or L after the string number instead of - a colon. This is in addition to the normal full list. The string length - (that is, the return from the extraction function) is given in paren- - theses after each string for \C and \G. - - Note that whereas patterns can be continued over several lines (a plain - ">" prompt is used for continuations), data lines may not. However new- - lines can be included in data by means of the \n escape (or \r, \r\n, - etc., depending on the newline sequence setting). - - -OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION - - When the alternative matching function, pcre[16|32]_dfa_exec(), is used - (by means of the \D escape sequence or the -dfa command line option), - the output consists of a list of all the matches that start at the - first point in the subject where there is at least one match. For exam- - ple: - - re> /(tang|tangerine|tan)/ - data> yellow tangerine\D - 0: tangerine - 1: tang - 2: tan - - (Using the normal matching function on this data finds only "tang".) - The longest matching string is always given first (and numbered zero). - After a PCRE_ERROR_PARTIAL return, the output is "Partial match:", fol- - lowed by the partially matching substring. (Note that this is the - entire substring that was inspected during the partial match; it may - include characters before the actual match start if a lookbehind asser- - tion, \K, \b, or \B was involved.) - - If /g is present on the pattern, the search for further matches resumes - at the end of the longest match. For example: - - re> /(tang|tangerine|tan)/g - data> yellow tangerine and tangy sultana\D - 0: tangerine - 1: tang - 2: tan - 0: tang - 1: tan - 0: tan - - Since the matching function does not support substring capture, the - escape sequences that are concerned with captured substrings are not - relevant. - - -RESTARTING AFTER A PARTIAL MATCH - - When the alternative matching function has given the PCRE_ERROR_PARTIAL - return, indicating that the subject partially matched the pattern, you - can restart the match with additional subject data by means of the \R - escape sequence. For example: - - re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ - data> 23ja\P\D - Partial match: 23ja - data> n05\R\D - 0: n05 - - For further information about partial matching, see the pcrepartial - documentation. - - -CALLOUTS - - If the pattern contains any callout requests, pcretest's callout func- - tion is called during matching. This works with both matching func- - tions. By default, the called function displays the callout number, the - start and current positions in the text at the callout time, and the - next pattern item to be tested. For example: - - --->pqrabcdef - 0 ^ ^ \d - - This output indicates that callout number 0 occurred for a match - attempt starting at the fourth character of the subject string, when - the pointer was at the seventh character of the data, and when the next - pattern item was \d. Just one circumflex is output if the start and - current positions are the same. - - Callouts numbered 255 are assumed to be automatic callouts, inserted as - a result of the /C pattern modifier. In this case, instead of showing - the callout number, the offset in the pattern, preceded by a plus, is - output. For example: - - re> /\d?[A-E]\*/C - data> E* - --->E* - +0 ^ \d? - +3 ^ [A-E] - +8 ^^ \* - +10 ^ ^ - 0: E* - - If a pattern contains (*MARK) items, an additional line is output when- - ever a change of latest mark is passed to the callout function. For - example: - - re> /a(*MARK:X)bc/C - data> abc - --->abc - +0 ^ a - +1 ^^ (*MARK:X) - +10 ^^ b - Latest Mark: X - +11 ^ ^ c - +12 ^ ^ - 0: abc - - The mark changes between matching "a" and "b", but stays the same for - the rest of the match, so nothing more is output. If, as a result of - backtracking, the mark reverts to being unset, the text "<unset>" is - output. - - The callout function in pcretest returns zero (carry on matching) by - default, but you can use a \C item in a data line (as described above) - to change this and other parameters of the callout. - - Inserting callouts can be helpful when using pcretest to check compli- - cated regular expressions. For further information about callouts, see - the pcrecallout documentation. - - -NON-PRINTING CHARACTERS - - When pcretest is outputting text in the compiled version of a pattern, - bytes other than 32-126 are always treated as non-printing characters - are are therefore shown as hex escapes. - - When pcretest is outputting text that is a matched part of a subject - string, it behaves in the same way, unless a different locale has been - set for the pattern (using the /L modifier). In this case, the - isprint() function to distinguish printing and non-printing characters. - - -SAVING AND RELOADING COMPILED PATTERNS - - The facilities described in this section are not available when the - POSIX interface to PCRE is being used, that is, when the /P pattern - modifier is specified. - - When the POSIX interface is not in use, you can cause pcretest to write - a compiled pattern to a file, by following the modifiers with > and a - file name. For example: - - /pattern/im >/some/file - - See the pcreprecompile documentation for a discussion about saving and - re-using compiled patterns. Note that if the pattern was successfully - studied with JIT optimization, the JIT data cannot be saved. - - The data that is written is binary. The first eight bytes are the - length of the compiled pattern data followed by the length of the - optional study data, each written as four bytes in big-endian order - (most significant byte first). If there is no study data (either the - pattern was not studied, or studying did not return any data), the sec- - ond length is zero. The lengths are followed by an exact copy of the - compiled pattern. If there is additional study data, this (excluding - any JIT data) follows immediately after the compiled pattern. After - writing the file, pcretest expects to read a new pattern. - - A saved pattern can be reloaded into pcretest by specifying < and a - file name instead of a pattern. There must be no space between < and - the file name, which must not contain a < character, as otherwise - pcretest will interpret the line as a pattern delimited by < charac- - ters. For example: - - re> </some/file - Compiled pattern loaded from /some/file - No study data - - If the pattern was previously studied with the JIT optimization, the - JIT information cannot be saved and restored, and so is lost. When the - pattern has been loaded, pcretest proceeds to read data lines in the - usual way. - - You can copy a file written by pcretest to a different host and reload - it there, even if the new host has opposite endianness to the one on - which the pattern was compiled. For example, you can compile on an i86 - machine and run on a SPARC machine. When a pattern is reloaded on a - host with different endianness, the confirmation message is changed to: - - Compiled pattern (byte-inverted) loaded from /some/file - - The test suite contains some saved pre-compiled patterns with different - endianness. These are reloaded using "<!" instead of just "<". This - suppresses the "(byte-inverted)" text so that the output is the same on - all hosts. It also forces debugging output once the pattern has been - reloaded. - - File names for saving and reloading can be absolute or relative, but - note that the shell facility of expanding a file name that starts with - a tilde (~) is not available. - - The ability to save and reload files in pcretest is intended for test- - ing and experimentation. It is not intended for production use because - only a single pattern can be written to a file. Furthermore, there is - no facility for supplying custom character tables for use with a - reloaded pattern. If the original pattern was compiled with custom - tables, an attempt to match a subject string using a reloaded pattern - is likely to cause pcretest to crash. Finally, if you attempt to load - a file that is not in the correct format, the result is undefined. - - -SEE ALSO - - pcre(3), pcre16(3), pcre32(3), pcreapi(3), pcrecallout(3), pcrejit, - pcrematching(3), pcrepartial(d), pcrepattern(3), pcreprecompile(3). - - -AUTHOR - - Philip Hazel - University Computing Service - Cambridge CB2 3QH, England. - - -REVISION - - Last updated: 09 February 2014 - Copyright (c) 1997-2014 University of Cambridge. diff --git a/plugins/Pcre16/docs/doc/pcreunicode.3 b/plugins/Pcre16/docs/doc/pcreunicode.3 deleted file mode 100644 index cb5e5269a4..0000000000 --- a/plugins/Pcre16/docs/doc/pcreunicode.3 +++ /dev/null @@ -1,249 +0,0 @@ -.TH PCREUNICODE 3 "27 February 2013" "PCRE 8.33" -.SH NAME -PCRE - Perl-compatible regular expressions -.SH "UTF-8, UTF-16, UTF-32, AND UNICODE PROPERTY SUPPORT" -.rs -.sp -As well as UTF-8 support, PCRE also supports UTF-16 (from release 8.30) and -UTF-32 (from release 8.32), by means of two additional libraries. They can be -built as well as, or instead of, the 8-bit library. -. -. -.SH "UTF-8 SUPPORT" -.rs -.sp -In order process UTF-8 strings, you must build PCRE's 8-bit library with UTF -support, and, in addition, you must call -.\" HREF -\fBpcre_compile()\fP -.\" -with the PCRE_UTF8 option flag, or the pattern must start with the sequence -(*UTF8) or (*UTF). When either of these is the case, both the pattern and any -subject strings that are matched against it are treated as UTF-8 strings -instead of strings of individual 1-byte characters. -. -. -.SH "UTF-16 AND UTF-32 SUPPORT" -.rs -.sp -In order process UTF-16 or UTF-32 strings, you must build PCRE's 16-bit or -32-bit library with UTF support, and, in addition, you must call -.\" HREF -\fBpcre16_compile()\fP -.\" -or -.\" HREF -\fBpcre32_compile()\fP -.\" -with the PCRE_UTF16 or PCRE_UTF32 option flag, as appropriate. Alternatively, -the pattern must start with the sequence (*UTF16), (*UTF32), as appropriate, or -(*UTF), which can be used with either library. When UTF mode is set, both the -pattern and any subject strings that are matched against it are treated as -UTF-16 or UTF-32 strings instead of strings of individual 16-bit or 32-bit -characters. -. -. -.SH "UTF SUPPORT OVERHEAD" -.rs -.sp -If you compile PCRE with UTF support, but do not use it at run time, the -library will be a bit bigger, but the additional run time overhead is limited -to testing the PCRE_UTF[8|16|32] flag occasionally, so should not be very big. -. -. -.SH "UNICODE PROPERTY SUPPORT" -.rs -.sp -If PCRE is built with Unicode character property support (which implies UTF -support), the escape sequences \ep{..}, \eP{..}, and \eX can be used. -The available properties that can be tested are limited to the general -category properties such as Lu for an upper case letter or Nd for a decimal -number, the Unicode script names such as Arabic or Han, and the derived -properties Any and L&. Full lists is given in the -.\" HREF -\fBpcrepattern\fP -.\" -and -.\" HREF -\fBpcresyntax\fP -.\" -documentation. Only the short names for properties are supported. For example, -\ep{L} matches a letter. Its Perl synonym, \ep{Letter}, is not supported. -Furthermore, in Perl, many properties may optionally be prefixed by "Is", for -compatibility with Perl 5.6. PCRE does not support this. -. -. -.\" HTML <a name="utf8strings"></a> -.SS "Validity of UTF-8 strings" -.rs -.sp -When you set the PCRE_UTF8 flag, the byte strings passed as patterns and -subjects are (by default) checked for validity on entry to the relevant -functions. The entire string is checked before any other processing takes -place. From release 7.3 of PCRE, the check is according the rules of RFC 3629, -which are themselves derived from the Unicode specification. Earlier releases -of PCRE followed the rules of RFC 2279, which allows the full range of 31-bit -values (0 to 0x7FFFFFFF). The current check allows only values in the range U+0 -to U+10FFFF, excluding the surrogate area. (From release 8.33 the so-called -"non-character" code points are no longer excluded because Unicode corrigendum -#9 makes it clear that they should not be.) -.P -Characters in the "Surrogate Area" of Unicode are reserved for use by UTF-16, -where they are used in pairs to encode codepoints with values greater than -0xFFFF. The code points that are encoded by UTF-16 pairs are available -independently in the UTF-8 and UTF-32 encodings. (In other words, the whole -surrogate thing is a fudge for UTF-16 which unfortunately messes up UTF-8 and -UTF-32.) -.P -If an invalid UTF-8 string is passed to PCRE, an error return is given. At -compile time, the only additional information is the offset to the first byte -of the failing character. The run-time functions \fBpcre_exec()\fP and -\fBpcre_dfa_exec()\fP also pass back this information, as well as a more -detailed reason code if the caller has provided memory in which to do this. -.P -In some situations, you may already know that your strings are valid, and -therefore want to skip these checks in order to improve performance, for -example in the case of a long subject string that is being scanned repeatedly. -If you set the PCRE_NO_UTF8_CHECK flag at compile time or at run time, PCRE -assumes that the pattern or subject it is given (respectively) contains only -valid UTF-8 codes. In this case, it does not diagnose an invalid UTF-8 string. -.P -Note that passing PCRE_NO_UTF8_CHECK to \fBpcre_compile()\fP just disables the -check for the pattern; it does not also apply to subject strings. If you want -to disable the check for a subject string you must pass this option to -\fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP. -.P -If you pass an invalid UTF-8 string when PCRE_NO_UTF8_CHECK is set, the result -is undefined and your program may crash. -. -. -.\" HTML <a name="utf16strings"></a> -.SS "Validity of UTF-16 strings" -.rs -.sp -When you set the PCRE_UTF16 flag, the strings of 16-bit data units that are -passed as patterns and subjects are (by default) checked for validity on entry -to the relevant functions. Values other than those in the surrogate range -U+D800 to U+DFFF are independent code points. Values in the surrogate range -must be used in pairs in the correct manner. -.P -If an invalid UTF-16 string is passed to PCRE, an error return is given. At -compile time, the only additional information is the offset to the first data -unit of the failing character. The run-time functions \fBpcre16_exec()\fP and -\fBpcre16_dfa_exec()\fP also pass back this information, as well as a more -detailed reason code if the caller has provided memory in which to do this. -.P -In some situations, you may already know that your strings are valid, and -therefore want to skip these checks in order to improve performance. If you set -the PCRE_NO_UTF16_CHECK flag at compile time or at run time, PCRE assumes that -the pattern or subject it is given (respectively) contains only valid UTF-16 -sequences. In this case, it does not diagnose an invalid UTF-16 string. -However, if an invalid string is passed, the result is undefined. -. -. -.\" HTML <a name="utf32strings"></a> -.SS "Validity of UTF-32 strings" -.rs -.sp -When you set the PCRE_UTF32 flag, the strings of 32-bit data units that are -passed as patterns and subjects are (by default) checked for validity on entry -to the relevant functions. This check allows only values in the range U+0 -to U+10FFFF, excluding the surrogate area U+D800 to U+DFFF. -.P -If an invalid UTF-32 string is passed to PCRE, an error return is given. At -compile time, the only additional information is the offset to the first data -unit of the failing character. The run-time functions \fBpcre32_exec()\fP and -\fBpcre32_dfa_exec()\fP also pass back this information, as well as a more -detailed reason code if the caller has provided memory in which to do this. -.P -In some situations, you may already know that your strings are valid, and -therefore want to skip these checks in order to improve performance. If you set -the PCRE_NO_UTF32_CHECK flag at compile time or at run time, PCRE assumes that -the pattern or subject it is given (respectively) contains only valid UTF-32 -sequences. In this case, it does not diagnose an invalid UTF-32 string. -However, if an invalid string is passed, the result is undefined. -. -. -.SS "General comments about UTF modes" -.rs -.sp -1. Codepoints less than 256 can be specified in patterns by either braced or -unbraced hexadecimal escape sequences (for example, \ex{b3} or \exb3). Larger -values have to use braced sequences. -.P -2. Octal numbers up to \e777 are recognized, and in UTF-8 mode they match -two-byte characters for values greater than \e177. -.P -3. Repeat quantifiers apply to complete UTF characters, not to individual -data units, for example: \ex{100}{3}. -.P -4. The dot metacharacter matches one UTF character instead of a single data -unit. -.P -5. The escape sequence \eC can be used to match a single byte in UTF-8 mode, or -a single 16-bit data unit in UTF-16 mode, or a single 32-bit data unit in -UTF-32 mode, but its use can lead to some strange effects because it breaks up -multi-unit characters (see the description of \eC in the -.\" HREF -\fBpcrepattern\fP -.\" -documentation). The use of \eC is not supported in the alternative matching -function \fBpcre[16|32]_dfa_exec()\fP, nor is it supported in UTF mode by the -JIT optimization of \fBpcre[16|32]_exec()\fP. If JIT optimization is requested -for a UTF pattern that contains \eC, it will not succeed, and so the matching -will be carried out by the normal interpretive function. -.P -6. The character escapes \eb, \eB, \ed, \eD, \es, \eS, \ew, and \eW correctly -test characters of any code value, but, by default, the characters that PCRE -recognizes as digits, spaces, or word characters remain the same set as in -non-UTF mode, all with values less than 256. This remains true even when PCRE -is built to include Unicode property support, because to do otherwise would -slow down PCRE in many common cases. Note in particular that this applies to -\eb and \eB, because they are defined in terms of \ew and \eW. If you really -want to test for a wider sense of, say, "digit", you can use explicit Unicode -property tests such as \ep{Nd}. Alternatively, if you set the PCRE_UCP option, -the way that the character escapes work is changed so that Unicode properties -are used to determine which characters match. There are more details in the -section on -.\" HTML <a href="pcrepattern.html#genericchartypes"> -.\" </a> -generic character types -.\" -in the -.\" HREF -\fBpcrepattern\fP -.\" -documentation. -.P -7. Similarly, characters that match the POSIX named character classes are all -low-valued characters, unless the PCRE_UCP option is set. -.P -8. However, the horizontal and vertical white space matching escapes (\eh, \eH, -\ev, and \eV) do match all the appropriate Unicode characters, whether or not -PCRE_UCP is set. -.P -9. Case-insensitive matching applies only to characters whose values are less -than 128, unless PCRE is built with Unicode property support. A few Unicode -characters such as Greek sigma have more than two codepoints that are -case-equivalent. Up to and including PCRE release 8.31, only one-to-one case -mappings were supported, but later releases (with Unicode property support) do -treat as case-equivalent all versions of characters such as Greek sigma. -. -. -.SH AUTHOR -.rs -.sp -.nf -Philip Hazel -University Computing Service -Cambridge CB2 3QH, England. -.fi -. -. -.SH REVISION -.rs -.sp -.nf -Last updated: 27 February 2013 -Copyright (c) 1997-2013 University of Cambridge. -.fi diff --git a/plugins/Pcre16/docs/doc/perltest.txt b/plugins/Pcre16/docs/doc/perltest.txt deleted file mode 100644 index bb1a52a48e..0000000000 --- a/plugins/Pcre16/docs/doc/perltest.txt +++ /dev/null @@ -1,42 +0,0 @@ -The perltest program --------------------- - -The perltest.pl script tests Perl's regular expressions; it has the same -specification as pcretest, and so can be given identical input, except that -input patterns can be followed only by Perl's lower case modifiers and certain -other pcretest modifiers that are either handled or ignored: - - /+ recognized and handled by perltest - /++ the second + is ignored - /8 recognized and handled by perltest - /J ignored - /K ignored - /W ignored - /S ignored - /SS ignored - /Y ignored - -The pcretest \Y escape in data lines is removed before matching. The data lines -are processed as Perl double-quoted strings, so if they contain " $ or @ -characters, these have to be escaped. For this reason, all such characters in -the Perl-compatible testinput1 file are escaped so that they can be used for -perltest as well as for pcretest. The special upper case pattern modifiers such -as /A that pcretest recognizes, and its special data line escapes, are not used -in the Perl-compatible test file. The output should be identical, apart from -the initial identifying banner. - -The perltest.pl script can also test UTF-8 features. It recognizes the special -modifier /8 that pcretest uses to invoke UTF-8 functionality. The testinput4 -and testinput6 files can be fed to perltest to run compatible UTF-8 tests. -However, it is necessary to add "use utf8; require Encode" to the script to -make this work correctly. I have not managed to find a way to handle this -automatically. - -The other testinput files are not suitable for feeding to perltest.pl, since -they make use of the special upper case modifiers and escapes that pcretest -uses to test certain features of PCRE. Some of these files also contain -malformed regular expressions, in order to check that PCRE diagnoses them -correctly. - -Philip Hazel -January 2012 |