#1501 Release of dune 2.4

Metadata

Property	Value
Reported by	Carsten Gräser (graeser@math.fu-berlin.de)
Reported at	Sep 29, 2014 10:13
Type	Discussion
Version	Git (pre2.4) [autotools]
Operating System	Unspecified / All
Last edited by	Carsten Gräser (graeser@math.fu-berlin.de)
Last edited at	Oct 6, 2015 09:12
Closed by	Carsten Gräser (graeser@math.fu-berlin.de)
Closed at	Oct 6, 2015 09:12
Closed in version	Unknown
Resolution	Implemented
Comment	Release has been published.

Description

This is for collecting stuff related to the 2.4 release.

On the 2014 dev meeting the following features have been voted for 2.4:

• 2. Add all (despite well-founded exceptions) flags by default. (Markus)

The following have been voted w/o dedicated release number but may also be suited for 2.4:

• 21. Get rid of istl_assign_to_fmatrix (Christoph Ge.)

The followings features have been voted for 2.4 but where skipped for the release:

• 11. Deprecate overlapSize(), ghostSize(), communicate() on grid.
• 17. Add hasEntityIterator capability
• 24. Remove C++11 compatibility code

The followings task are done:

• 1. Make cmake the default and add --use-autotools in dunecontrol.
• 3. Deprecate SGrid (Oliver)
• 4. Remove Alberta 2 support.
• 5. Merge all-entities-yaspgrid.
• 6. Add global index set from Benedikt Oswald and check compatibility with scotch.
• 7. Make EntityIterators forward_iterators.
• 8. Make entities copyable.
• 9. Make intersections copyable.
• 10. Deprecate PartitionIteratorType paramter of gridviews (Martin)
• 12. Deprecate IndexSet::geomTypes()
• 13. Add types() on indexsets and gridviews (Martin)
• 14. Document and merge range based for (Steffen)
• 15. Deprecate map() and add index() subIndex() to mappers (Oliver)
• 16. Use Mapper::IndexType instead of plain int (Oliver)
• 18. Make gridcheck headers .hh (Steffen)
• 19. Rename monom.hh to monomial.hh (Oliver)
• 20. Only run boost check in dune-common if necessary.
• 22. Deprecate Geometry::dimension and Geometry::dimensionworld (Oliver)
• 23. Deprecate isParallel capability

I'd also like to have make_unique in 2.4.

13. is done

14. is done as far as possible (in my opinion)

8. and 9) are implemented in the facade classes and for YaspGrid.

4. is done. 19) was done recently by Oliver, too.

I'd like to add #1435 (closed) as a dependency, so we can properly deprecate Dune::shared_ptr et al.

I added 1435 (we already decided on this on the 2013 dev meeting).

I just noticed that 'types()' and 'Types' are missing on GridView, so 13) is still party open.

• I forgot to mention: I did 20) last week.

• The problem with #1435 (closed) is, that I cannot deprecate the headers importing std::shared_ptr & co. into Dune::, as other, probably useful functions and classes are part of the headers. Nobody answered so far what should be done with this helper code.

• According to the protocol, Martin wanted to remove lbegin() and lend() from the interface and the gridcheck. Is this for 2.4, too?

• Carsten, will you fix #980 (closed) for the release?

• Together with Dominic, we fixed the PKG_ALL mechanism for CMake (except Alberta and dune-alugrid, I'll have a look within the next weeks). I asked Markus and he won't be able to do more for the 2.4 release. Thus this is partially done and the rest is postponed to Dune 3.0.

• I guess 24) won't be in Dune 2.4 as nobody seems willing to discuss what should be deprecated and what should be kept.

• I think we should consider #1511 (closed) - Copyable entities and intersections. From my point of view, without the transition code from Steffen, we'll have a hard time fixing all the breaking code in Dune 3.0. Having hundreds of breaking occurrences will scare of people, similar to the Python 3 debacle.

As I have to touch the gridcheck anyway (for the copyable entities stuff), I went ahead and renamed the gridcheck headers. The old files are still there, but raise a deprecation warning.

So what's the status? The original plan was to release at the end of last year. Do we have a new schedule? How about simply releasing now and leaving the remaining features for the next release?

Branching now and releasing in six weeks seems fine to me. That would be enough time to give 2.4 the final polish and would be a clear statement to actually do the polishing.

So a release on about May 20. Sounds good. Any objections? Christoph, can you create the branches?

Can you wait until tonight? I have a fix for the buildsystem issues with dune_enable_all_packages and libraries that I'd like to get in before the branch...

I will wait until Monday with branching. This gives others the chance to object. If Carsten wants to take over, he can do so.

OK, since there where no objections, we should drop the remaining open issues. I will push the branches without further checking to fix the current state.

Any bug-fixing can then be done on this branch.

I just pushed the branches. Happy testing and bug-fixing.

Please do larger bug fixes in separate branches based on the releases/2.4 from now on. Then we can merge them easily into release and master without cherry-picking, even if master starts deviating.

Please merge feature/fix-variablesizecom-with-empty-interface

I added a nightly build for the 2.4 release branch (with and without cmake) to m build server. Will report problems here. HTH

Mmh, there seems to be a release branch for the grid-howto missing.

I added the release branch to grid-howto and I back-ported all commits that I committed to master, too. Carsten, if I interfere to much with your release manager duties, please stop me.

@carsten: How is cherry-picking handled? Do you do it or should we do it ourselves? Last time I had a script that scanned the master for release or bugfix tags, see https://github.com/blattms/git-release-tools

@Christoph: Thanks for making the branch.

@Markus: As outlined earlier I would appreciate to have fixes in a branch. This makes tracking merges easier. However single-commit fixes are surely better cherry-picked. I'll take care about this, but please mark your commits as outlined by Markus. BTW: Thanks for sharing the script.

I am afraid I have to voice a certain discontent regarding the release process. Hardly anything happens on the release branch. Ansgar is waiting for a few patches to be cherry-picked (see his mail on dune-devel from Apr.30), to be able to start Debian packaging. Is anybody taking care of these patches? Previously, standard procedure allowed only the release manager to push to the release branches. Is that still valid?

Hi Oliver,

the patches I was waiting for were merged earlier today.

Ansgar

What is the plan for Dune 2.4? Can we decide which tasks are really release critical? The road-map [1] and the list of dependent tasks contain pretty old bugs nobody cares. I propose to postpone all these tasks to Dune 2.4.1 or 3.0 unless someone volunteers (or gets volunteered...) to fix them within the next two week. Then we can create a release candidate and finally get the release out of the door.

[1] https://dune-project.org/flyspray/index.php?do=roadmap&project=1

Carsten would you be so kind backporting two commits I did not properly tag? dune-common dfeeea798b3f2360c45407ac9172c9a69dc46da9 and 2d38fedf758cd58e61b5d6b908bd68890e97e08d

So when are we getting the first release candidates?

@Christoph: Done.

Carsten, can you pack-port 66219680ebc05993a55ba6fa98d8fbbc4395cc4f? I even consider pack-porting Robert's tmpnam patch to Dune 2.3.

@Christoph: I already did.

Nicolas Schwenck (nicolas.schwenck@iws.uni-stuttgart.de) commented on the issue:

Dear Dune-developers

As acting release manager for the Dumux-2.8 release, I am wondering if there exists an advanced schedule when to release DUNE-2.4?

It would be really nice for us to build upon, but if there is no actual estimated release date, we have to rely on DUNE-2.3.1. Or is there maybe a release candidate already out there somewhere?

Kind regards, Nicolas

There is, I want to merge the remaining merge requests and then prepare the tarballs.

BTW: Now that autotools are deprecated, what's the desired way to build tarballs?

You have to build the tarballs still with Autotools. Otherwise they won't contain the necessary magic (Makefile.in's etc.).

How does the 'otherwise' option work? Do I have to call cpack?

OK, I merged all requested commits. Anything not in 2.4 will have to wait now.

Otherwise is with CMake / CPack. Have a loot at #1607 (closed) - CMake based tarballs. But then Autotools users need the Autotools to configure Dune, similar to a Git checkout. I don't care which way you choose.

Nicolas Schwenck (nicolas.schwenck@iws.uni-stuttgart.de) commented on the issue:

Is there any news on a release tarball?

I just tagged rc1. I'll upload the tarballs once I've found out how to do this.

One important missing step for 2.4 is updating the homepage. Currently the installation notes list autotools as requirement.

Great news, thank you.

Just in case you did not yet figure out how to upload the tarballs: There is a release manager guide in the user wiki [1]. Please update the guide if you find any errors our outdated advices.

[1] https://users.dune-project.org/projects/main-wiki/wiki/Guides_release_manager

I know, that you compiled this page but could not find it on the core module wiki. Thanks for the link.

I built the git tags on openSUSE 13.2 with gcc 4.7, 4.8, 4.9, 5.1 and clang 3.5. MPI is openmpi 1.8.3. My third-party library setup is attached. I ran ~100 Dumux tests distributed over and on top of the different compiler builds. All fine. Thank you!

Attachments

• bernd_setup.txt

I just tested the RC1 tarballs with two configurations:

1. clang 3.6.1 with CMake 2.8.6 (oldest supported version)

• the old CMake did not find my LAPACK which triggered eigenvaluetest to be not linked correctly. I fixed it in master, please backport the two commits a46de680247ed670372d0f39ac8d26718c6d02bd and 635500eb1a41381fb88386b43b8b0003e8eee425
• the old CMake did have some troubles to add the ParMETIS flags to the test in dune-istl. I don't know what's the problem.
• There is a dgf missing in the dune-grid tarball and CMake passed the wrong grid dim. Please backport 1590d83b73546347055bbb08dfbd268758d8a25a
• types passed via pre-processor macros where chopped at spaces. not sure whether this is a problem from the old CMake, the new Clang or a combination of both. Fixed it anyway: 3fba2fbf457e593255bb79879c77334dc3504da7
• dune-grid-howto does not get the flags passed as we use code that is not compatible with CMake 2.8.6. Maybe we should consider backporting 4afbc497f532a4d10ddc18b419f18ca6d2a57b77 and require for dune-grid-howto a newer CMake version

2. Clang 3.7 release branch (a couple of commits after RC1) with CMake 3.3~RC4.

• Just works with the changes made above.

Just for the sake of completeness: Configuration 2b) like (2) but with Ninja: Works flawlessly, too.

@Bernd: Thanks for the testing. That's good news.

@Christoph: I'll take care of picking the mentioned commits.

Regarding the ParMETIS issue: What does 'CMAke did have some troubles' mean?

Regarding grid-howto: If we support cmake 2.8.6 this should in principle also be true for the tutorial. Would it be complicated to make this compatible? If yes: Maybe it's OK to require a newer cmake, because people will normally use the tutorial when they get started on their desktop computer (with hopefully newer setup) and not on their cluster with conservative update policies.

OK, I updated dune-common, dune-geometry, dune-istl, dune-localfunctions. Patches to dune-grid are currently being tested and will hopefully be pushed soon.

Any other opinions on requiring a newer cmake for dune-grid-howto? @Christoph: Which version would we need?

I removed the following dependencies for 2.4:

#1212 (closed): This seems to need more investigation. Since the proposed patch was not merged to 2.4 (but to master), we will not support metis 5 in 2.4.

#1706 (closed): The proposed patch was not even merged

#1695 (closed): This is unlikely to get fixed. Dune-grid will not work with UG and gcc-4.4.

According to dune-grid-howto's CMakeLists.txt we require CMake 2.8.12. It can be back-ported to CMake 2.8.6 within reasonable amount of time. I have flinched from doing so because testing all combinations of grids is so time consuming.

@Christoph: Then I'd rather keep it as a known issue.

What's the schedule for the release? Is there a blocker left? Will we get a release this week?

Yes we will, once I've checked the last minute request from Steffen.

Could we release on Sunday/Monday in coordination with the user meeting? Then we have a release party, anyway.

@Markus: Do you want me to wait until this or are you asking if I'll be done until this?

Current status: After merging Steffens request and another small patch all tests run with the default compiler. I'm just waiting for all the other compilers. If there are no regressions, we're done.

Great news! Please don't wait any longer, there will be enough joy left to party a whole week.

Done.

Status changed to closed