dune-common merge requestshttps://gitlab.dune-project.org/core/dune-common/-/merge_requests2022-05-03T11:45:03Zhttps://gitlab.dune-project.org/core/dune-common/-/merge_requests/1080WIP: [ParameterTreeParser] add ParameterTreeParser for python scripts2022-05-03T11:45:03ZNils-Arne DreierWIP: [ParameterTreeParser] add ParameterTreeParser for python scriptsThis MR adds two static methods to `ParameterTreeParser` that can be used to read a python dictionary into a ParameterTree.
`readPythonDict`:
- Traverses all key-value pairs in the python dictionary
- if the value is an instance of ...This MR adds two static methods to `ParameterTreeParser` that can be used to read a python dictionary into a ParameterTree.
`readPythonDict`:
- Traverses all key-value pairs in the python dictionary
- if the value is an instance of `pybind11::dict` readPythonDict is called for the sub-ParameterTree of the corresponding key
- else the string representation is added to the ParameterTree for the corresponding key
`readPythonFile`:
- The python script is interpret using pybind11 `eval_file` method
- The variable with the specified name (default: `__dict__`, which is the global scope) is casted to a `pybind11:dict` and passed to `readPythonDict`
For `readPythonDict` the include files and libraries of python must be available. I added a CMake variable `HAVE_PYTHON3_DEV` and an entry in the `config.h` for that. Futhermore I added `AddPythonFlags.cmake` for the management of the linking and includes.
For `readPythonFile` the embed functionality of python must be available this is checked with the CMake variable `Python3_Development.Embed_FOUND` and passed to the c++ side with `HAVE_PYTHON3_EMBED`.
Furthermore a `type_caster` is added for conversion of a python dict into a ParameterTree. That makes python bindings that take a ParameterTree as a function argument callable from the python-side with a dict.ParameterTree overhaulthttps://gitlab.dune-project.org/core/dune-common/-/merge_requests/564WIP: A generic string-to-number conversion function2022-04-04T15:27:54ZSimon PraetoriusWIP: A generic string-to-number conversion function### Summary
Add facility to cast from character sequence to any number type, implemented in terms of std library functions
like `atoi()` and `strf()` but parametrized with the concrete target type for ease of usage
in generic code.
### ...### Summary
Add facility to cast from character sequence to any number type, implemented in terms of std library functions
like `atoi()` and `strf()` but parametrized with the concrete target type for ease of usage
in generic code.
### Motivation
There is not generic string-to-number conversion function in the std library, except for `std::ostringstream`. But, the latter is quite slow compared to type specific functions like `strtof()` (about a factor of 10 slower) and needs about 3 lines of code compared to a direct conversion function. In other libraries, like boost, generic functions for string-to-number conversions are added that are much faster that the intrinsics. Here a very simple approach is used to get a generic converter, but simply specializing a helper function on the concrete number type and calling the type-specific std function.
### Application
I have in mind the conversion of quadrature points and weights in dune-geometry from a string representation of the number, to allow high-precision data to be used.
### Discussion
### Locale independent implementation
When using std library functions, like `std::strtof` these are locale dependent, meaning that e.g. the decimal dot may be different on different systems leading to different behavior of the code. Do we need locale independent conversion, e.g. for parsing ini files or for converting stored quadrature values? Or is the user responsible for setting the correct locale setting in its main program?
- Locale independent conversion is currently only possible with `std::stringstream`, with c++17 we may have `from_chars` but there is no working implementation in any std library, currently.
- Locale dependent implementation can be done with any strtof like functions. Additionaly, a check can be implemented that throws an error if the string can not be parsed correctly in the current locale setting.
#### Alternatives
The simplest alternative is `std::ostringstream`, but as explained above, it has the drawback of slow performance and not so easy to use. A second alternative is `ParameterTree::Parser<T>::parse(...)`, but it is a private implementation details of the `ParameterTree` class and uses `std::stringstream` internally.
The usage of these std library functions has the drawback of little error safety, i.e. for `atoi`
the default behavior is: If the converted value falls out of range of corresponding return type,
the return value is undefined. If no conversion can be performed, 0 is returned. But on the other
hand it gives high performance in conversion, what is intended here.
#### Naming
Alternative feature names could be considered:
- `lexicalCast` (like `boost::lexical_cast`)
- `cast`
- `toNumber` (inversion of `std::to_string`)
- `toArithmetic`
- `fromString`
- `fromChars` (similar to `std::from_chars` but with different signature)
- `parseString`
- `parse` (like `ParameterTree::Parser<T>::parse`)
- `strToNumber` (like `std::strtof`)
### TODO
* [ ] Replace test that uses random number, with a deterministic test of corner casesParameterTree overhaultJö Fahlkejorrit@jorrit.deJö Fahlkejorrit@jorrit.dehttps://gitlab.dune-project.org/core/dune-common/-/merge_requests/249WIP: Replace parameter tree parser with a new parser2022-11-11T21:20:44ZElias PippingWIP: Replace parameter tree parser with a new parserIt comes with two test suites: The old suite, which is available via
```
parametertreetest
```
and tests general parameter tree functionality, as well as a new test suite available via
```
iniparsertest
```
which is entirely dedicate...It comes with two test suites: The old suite, which is available via
```
parametertreetest
```
and tests general parameter tree functionality, as well as a new test suite available via
```
iniparsertest
```
which is entirely dedicated to parsing.
#### The language that the parser accepts is:
- Each line either contains no content, a scope, or an assignment (not both). A line ends at the first newline character. There is one exception: An assignment with a quoted string may contain multiple newlines.
- Each of the above three may be followed by a comment, which starts with a `#`. It may not be followed by anything else.
- A scope is made up of the character `[`, a prefix, and the character `]`. The prefix is a (potentially zero-length) string made up of `[a-zA-Z0-9._+- \t]` (that's what the parser sees. it's easy to restrict that further, but that can happen outside of the parser). Each of the three aforementioned tokens may be preceded or followed by an arbitrary amount of whitespace, i.e. `[ \t]`. Leading and trailing whitespace are stripped from the prefix.
- An assignment is made up of a key (a non-empty string made up of `[a-zA-Z0-9._+- \t]`; again, it would be easy to restrict this further from outside the parser), the character `=`, and a right-hand side. Each of these three tokens may be preceded or followed by an arbitrary amount of whitespace. Leading and trailing whitespace are stripped from the key. The right-hand side can be either of the following three: A simple string, a single-quoted string, or a double-quoted string.
- A simple string is a (potentially zero-length) sequence of characters of any character other than `[#'"\\]`. Leading and trailing whitespace are ignored.
- A single-quoted string starts and ends with a single quote: `'`. Inside, any character is allowed and treated as literal, except for two characters: `\` and `'`. A single `'` may not appear: It needs to be escaped like this: `\'`. A single `\` may not appear: it needs to be escaped like this: `\\`. The special sequence `\n` is treated as a newline character. Escaping any other character by preceding it with a backslash is an error. In addition to the special sequence `\n`, a single-quoted string may contain actual newline characters. They are preserved.
- Double-quoted strings behave exactly like single-quoted strings with `"` in place of `'`. Thus, inside such a string, `"` needs to be escaped but `'` may not.
The main deviations from the current parser I see, are thus:
- You can now put comments everywhere. The `#` character is treated properly inside and outside of strings.
- Multiline strings now reliably preserve newlines.
- The selection of admissible characters for a prefix is much narrower, maybe too narrow.
- The selection of admissible characters for a key is much narrower, maybe too narrow.ParameterTree overhault