[flang][NFC] Add F2023X documentation

[klausler]

Summary

Add a document that summarizes Fortran 202X's upcoming
features and their urgency for implementation.

[everythingfunctional]

flang/docs/F202X.md

91–96

I'd be curious to know if you have any examples of real world code that rely on the length of a buffer not changing, or would have meaningful different observable behaviour. In my experience the typical usage would be more like:

character(:), allocatable :: buffer
allocate(character(20)::buffer)
write(buffer,'F5.3') 3.14159
print *, trim(buffer)

which wouldn't show observably different behavior. Or at worst:

character(:), allocatable :: buffer
allocate(character(20)::buffer)
write(buffer,'F5.3') 3.14159
print *, buffer

where it is observably different (no more trailing blanks), but is likely to be a welcome change, not an unwelcome one.

[klausler]

flang/docs/F202X.md

91–96

I'm going to have to add portability warnings for usage of deferred-length character allocatables in these contexts and see how often they are triggered by nonclassified codes in order to give you an accurate reproducible answer to that. Part of the difficulty here is that this isn't usage that can be easily grepped for.

[everythingfunctional]

flang/docs/F202X.md

91–96

I think you're going to get a lot of false positives that way. I suspect the majority of cases will be from code that looks like:

character(:), allocatable :: buffer
allocate(character(1000) :: buffer)
write(buffer, 'F5.3') 3.14159
buffer = trim(buffer)

and are going to go, oh good, I can delete those two lines and just write:

character(:), allocatable :: buffer
write(buffer, 'F5.3') 3.14159

Now how do I turn the warning off? I'll tell you right now I'm in that camp.

My reading of your answer is either:

• "I don't know of any code that will actually break, but I'd like to scare everybody anyway", or
• "I know of some code that will break, but I'm not going to go to the effort of telling you what the problem actually is"

[kiranchandramohan]

Thanks for putting up this document. Very informative and likely good feedback for standards committee members. I see that Brad has already commented, may be Mark and Gary also might be interested.

A comment on the source line changes, relaxation for BOZ constants, using rank-one arrays to specify the rank and bounds of arrays might be useful.

flang/docs/F202X.md

141

There might be some performance benefits if lowering can use llvm attributes like argmem/argmemonly for simple functions.
https://github.com/llvm/llvm-project/blob/main/llvm/docs/Frontend/PerformanceTips.rst#modeling-memory-effects

199

Is Select case support for enumeration types new?

[klausler]

In D153916#4454929, @kiranchandramohan wrote:

Thanks for putting up this document. Very informative and likely good feedback for standards committee members. I see that Brad has already commented, may be Mark and Gary also might be interested.

A comment on the source line changes, relaxation for BOZ constants, using rank-one arrays to specify the rank and bounds of arrays might be useful.

F18 has always had no limit on free from source line length; are there other source line changes that I missed?

The BOZ literal changes originated in other implementations and they should already work in f18. Some compilers allow BOZ as output statement I/O data items, and that is intentionally not supported.

I did miss the thing about using vectors for bounds in array declarations; will read up on that and add it. Thanks for the tip!

flang/docs/F202X.md

141

My understanding is that calls to PURE procedures could use memory(read, argmem:readwrite, inaccessiblemem:readwrite) and SIMPLE could drop the read. Is that also what you have in mind?

Calls to SIMPLE procedures from DO CONCURRENT are also safe from the bug that feature has with PURE procedures.

199

It must be, since ENUMERATION TYPE is new in F202X.

[kiranchandramohan]

In D153916#4456315, @klausler wrote:

In D153916#4454929, @kiranchandramohan wrote:

Thanks for putting up this document. Very informative and likely good feedback for standards committee members. I see that Brad has already commented, may be Mark and Gary also might be interested.

A comment on the source line changes, relaxation for BOZ constants, using rank-one arrays to specify the rank and bounds of arrays might be useful.

F18 has always had no limit on free from source line length; are there other source line changes that I missed?

The BOZ literal changes originated in other implementations and they should already work in f18. Some compilers allow BOZ as output statement I/O data items, and that is intentionally not supported.

For both these (line-length and BOZ) I was thinking a statement like the above will be good for conveying the status of these features.

There is probably a requirement to provide warnings or errors if these limits are breached for source line length.
6.3.2.1 A line shall contain at most ten thousand characters.
6.3.2.6 A statement shall not have more than one million characters.

I did miss the thing about using vectors for bounds in array declarations; will read up on that and add it. Thanks for the tip!

flang/docs/F202X.md

141

Yes, I am hoping that the argument-only memory access might help enable more optimisation since it does not access any global memory.

We have seen in the past a case where there are two usages of the same pure function with the same arguments and it required classic-flang to provide the memory attributes to do a CSE.

bose_f in the following example.
https://github.com/yambo-code/yambo/blob/931e54cd54d721ca5cbbe8df3cfae4b6e2f1ffc8/src/qp/QP_ppa_cohsex.F#L622

[klausler]

In D153916#4459352, @kiranchandramohan wrote:

There is probably a requirement to provide warnings or errors if these limits are breached for source line length.
6.3.2.1 A line shall contain at most ten thousand characters.
6.3.2.6 A statement shall not have more than one million characters.

Fortran compilers are supposed to check numbered constraints and report on violations, but "shall"s in other text are optional. (And it's all on the honor system anyway -- there's no validation test suite or certification process.)

The 10,000 character limit per line would be easy to check, but the statement limit on characters would require some thought and would cost more than o(statements).

[klausler]

Edits to reflect review comments.

[kiranchandramohan]

Thanks.

flang/docs/F202X.md

+<!--===- docs/F202X.md 
+  
+   Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+   See https://llvm.org/LICENSE.txt for license information.
+   SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+  
+-->
+
+# A first take on Fortran 202X features for LLVM Flang
+
+I (Peter Klausler) have been studying the draft PDF of the
+[Fortran 202X standard](https://j3-fortran.org/doc/year/23/23-007r1.pdf),
+which will soon be published as ISO Fortran 2023.
+I have compiled this summary of its changes relative to
+the current Fortran 2018 standard from the perspective
+of a [Fortran compiler](https://github.com/llvm/llvm-project/tree/main/flang)
+implementor.
+
+## TL;DR
+
+Fortran 202X doesn't make very many changes to the language
+relative to Fortran 2018, which was itself a small increment
+over Fortran 2008.
+Apart from `REDUCE` clauses that were added to the
+[still broken](https://github.com/llvm/llvm-project/blob/main/flang/docs/DoConcurrent.md)
+`DO CONCURRENT` construct, there's little here for Fortran users
+to get excited about.
+
+## Priority of implementation in LLVM Flang
+
+We are working hard to ensure that existing working applications will
+port successfully to LLVM Flang with minimal effort.
+I am not particularly concerned with conforming to a new
+standard as an end in itself.
+
+The only features below that appear to have already been implemented
+in other compilers are the `REDUCE` clauses and the degree trigonometric
+intrinsic functions, so those should have priority as an aid to
+portability.
+We would want to support them earlier even if they were not in a standard.
+
+The `REDUCE` clause also merits early implementation due to
+its potential for performance improvements in real codes.
+I don't see any other feature here that would be relevant to
+performance (maybe a weak argument could be made for `SIMPLE`).
+The bulk of this revision unfortunately comprises changes to Fortran that
+are neither performance-related, already available in
+some compilers, nor (obviously) in use in existing codes.
+I will not prioritize implementing them myself over
+other work until they become portability concerns or are
+requested by actual users.
+
+Given Fortran's history of the latency between new
+standards and the support for their features in real compilers,
+and then the extra lag before the features are then actually used
+in codes meant to be portable, I doubt that many of the items
+below will have to be worked on any time soon due to user demand.
+
+If J3 had chosen to add more features that were material improvements
+to Fortran -- and there's quite a long list of worthy candidates that
+were passed over, like read-only pointers -- it would have made sense
+for me to prioritize their implementation in LLVM Flang more
+urgently.
+
+## Specific change descriptions
+
+The individual features added to the language are summarized
+in what I see as their order of significance to Fortran users.
+
+### Alert: There's a breaking change!
+
+The Fortran committee used to abhor making breaking changes,
+apart from fixes, so that conforming codes could be portable across
+time as well as across compilers.
+Fortran 202X, however, uncharacteristically perpetrates one such
+change to existing semantics that will silently cause existing
+codes to work differently, if that change were to be implemented
+and enabled by default.
+
+Specifically, automatic reallocation of whole deferred-length character
+allocatable scalars is now mandated when they appear for internal output
+(e.g., `WRITE(A,*) ...`)
+or as output arguments for some statements and intrinsic procedures
+(e.g., `IOMSG=`, `ERRMSG=`).
+So existing codes that allocate output buffers
+for such things will, or would, now observe that their buffers are
+silently changing their lengths during execution, rather than being
+padded with blanks or being truncated.  For example:
+
+```
+  character(:), allocatable :: buffer
+  allocate(character(20)::buffer)
+  write(buffer,'F5.3') 3.14159
+  print *, len(buffer)
+```
+
+prints 20 with Fortran 2018 but would print 5 with Fortran 202X.
+
+There would have no problem with the new standard changing the
+behavior in the current error case of an unallocated variable;
+defining new semantics for old errors is a generally safe means
+for extending a programming language.
+However, in this case, we'll need to protect existing conforming
+codes from the surprising new reallocation semantics, which
+affect cases that are not errors.
+
+When/if there are requests from real users to implement this breaking
+change, and if it is implemented, I'll have to ensure that users
+have the ability to control this change in behavior via an option &/or the
+runtime environment, and when it's enabled, emit a warning at code
+sites that are at risk.
+This warning should mention a source change they can make to protect
+themselves from this change by passing the complete substring (`A(:)`)
+instead of a whole character allocatable.
+
+This feature reminds me of Fortran 2003's change to whole
+allocatable array assignment, although in that case users were
+put at risk only of extra runtime overhead that was needless in
+existing codes, not a change in behavior, and users learned to
+assign to whole array sections (`A(:)=...`) rather than to whole
+allocatable arrays where the performance hit mattered.
+
+### Major Items
+
+The features in this section are expensive to implement in
+terms of engineering time to design, code, refactor, and test
+(i.e., weeks or months, not days).
+
+#### `DO CONCURRENT REDUCE`
+
+J3 continues to ignore the
+[serious semantic problems](https://github.com/llvm/llvm-project/blob/main/flang/docs/DoConcurrent.md)
+with `DO CONCURRENT`, despite the simplicity of the necessary fix and their
+admirable willingness to repair the standard to fix problems with
+other features (e.g., plugging holes in `PURE` procedure requirements)
+and their less admirable willingness to make breaking changes (see above).
+They did add `REDUCE` clauses to `DO CONCURRENT`, and those seem to be
+immediately useful to HPC codes and worth implementing soon.
+
+#### `SIMPLE` procedures
+
+The new `SIMPLE` procedures constitute a subset of F'95/HPF's `PURE`
+procedures.
+There are things that one can do in a `PURE` procedure
+but cannot in a `SIMPLE` one.  But the virtue of being `SIMPLE` seems
+to be its own reward, not a requirement to access any other
+feature.
+
+`SIMPLE` procedures might have been more useful had `DO CONCURRENT` been
+changed to require callees to be `SIMPLE`, not just `PURE`.
+
+The implementation of `SIMPLE` will be nontrivial: it involves
+some parsing and symbol table work, and some generalization of the
+predicate function `IsPureProcedure()`, extending the semantic checking on
+calls in `PURE` procedures to ensure that `SIMPLE` procedures
+only call other `SIMPLE` procedures, and modifying the intrinsic
+procedure table to note that most intrinsics are now `SIMPLE`
+rather than just `PURE`.
+
+I don't expect any codes to rush to change their `PURE` procedures
+to be `SIMPLE`, since it buys little and reduces portability.
+This makes `SIMPLE` a lower-priority feature.
+
+#### Conditional expressions and actual arguments
+
+Next on the list of "big ticket" items are C-style conditional
+expressions.  These come in two forms, each of which is a distinct
+feature that would be nontrivial to implement, and I would not be
+surprised to see some compilers implement one before the other.
+
+The first form is a new parenthesized expression primary that any C programmer
+would recognize.  It has straightforward parsing and semantics,
+but will require support in folding and all other code that
+processes expressions.  Lowering will be nontrivial due to
+control flow.
+
+The second form is a conditional actual argument syntax
+that allows runtime selection of argument associations, as well
+as a `.NIL.` syntax for optional arguments to signify an absent actual
+argument.  This would have been more useful if it had also been
+allowed as a pointer assignment statement right-hand side, and
+that might be a worthwhile extension.  As this form is essentially
+a conditional variable reference it may be cleaner to have a
+distinct representation from the conditional expression primary
+in the parse tree and strongly-typed `Expr<T>` representations.
+
+#### `ENUMERATION TYPE`
+
+Fortran 202X has a new category of type.  The new non-interoperable
+`ENUMERATION TYPE` feature is like C++'s `enum class` -- not, unfortunately,
+a powerful sum data type as in Haskell or Rust.  Unlike the
+current `ENUM, BIND(C)` feature, `ENUMERATION TYPE` defines a new
+type name and its distinct values.
+
+This feature may well be the item requiring the largest patch to
+the compiler for its implementation, as it affects parsing,
+type checking on assignment and argument association, generic
+resolution, formatted I/O, NAMELIST, debugging symbols, &c.
+It will indirectly affect every switch statement in the compiler
+that switches over the six (now seven) type categories.
+This will be a big project for little useful return to users.
+
+#### `TYPEOF` and `CLASSOF`
+
+Last on the list of "big ticket" items are the new TYPEOF and CLASSOF
+type specifiers, which allow declarations to indirectly use the
+types of previously-defined entities.  These would have obvious utility
+in a language with type polymorphism but aren't going to be very
+useful yet in Fortran 202X (esp. `TYPEOF`), although they would be worth
+supporting as a utility feature for a parametric module extension.
+
+`CLASSOF` has implications for semantics and lowering that need to
+be thought through as it seems to provide a means of
+declaring polymorphic local variables and function results that are
+neither allocatables nor pointers.
+
+#### Coarray extensions:
+
+ * `NOTIFY_TYPE`, `NOTIFY WAIT` statement, `NOTIFY=` specifier on image selector
+ * Arrays with coarray components
+
+#### "Rank Independent" Features
+
+The `RANK(n)` attribute declaration syntax is equivalent to
+`DIMENSION(:,:,...,:)` or an equivalent entity-decl containing `n` colons.
+As `n` must be a constant expression, that's straightforward to implement,
+though not terribly useful until the language acquires additional features.
+(I can see some utility in being able to declare PDT components with a
+`RANK` that depends on a `KIND` type parameter.)
+
+It is now possible to declare the lower and upper bounds of an explicit
+shape entity using a constant-length vector specification expression
+in a declaration, `ALLOCATE` statement, or pointer assignment with
+bounds remapping.
+For example, `real A([2,3])` is equivalent to `real A(2,3)`.
+
+The new `A(@V)` "multiple subscript" indexing syntax uses an integer
+vector to supply a list of subscripts or of triplet bounds/strides.  This one
+has tough edge cases for lowering that need to be thought through;
+for example, when the lengths of two or more of the vectors in
+`A(@U,@V,@W)` are not known at compilation time, implementing the indexing
+would be tricky in generated code and might just end up filling a
+temporary with `[U,V,W]` first.
+
+The obvious use case for "multiple subscripts" would be as a means to
+index into an assumed-rank dummy argument without the bother of a `SELECT RANK`
+construct, but that usage is not supported in Fortran 202X.
+
+This feature may well turn out to be Fortran 202X's analog to Fortran 2003's
+`LEN` derived type parameters.
+
+### Minor Items
+
+So much for the major features of Fortran 202X.  The longer list
+of minor features can be more briefly summarized.
+
+#### New Edit Descriptors
+
+Fortran 202X has some noncontroversial small tweaks to formatted output.
+The `AT` edit descriptor automatically trims character output.  The `LZP`,
+`LZS`, and `LZ` control edit descriptors and `LEADING_ZERO=` specifier provide a
+means for controlling the output of leading zero digits.
+
+#### Intrinsic Module Extensions
+
+Addressing some issues and omissions in intrinsic modules:
+
+ * LOGICAL8/16/32/64 and REAL16
+ * IEEE module facilities upgraded to match latest IEEE FP standard
+ * C_F_STRPOINTER, F_C_STRING for NUL-terminated strings
+ * C_F_POINTER(LOWER=)
+
+#### Intrinsic Procedure Extensions
+
+The `SYSTEM_CLOCK` intrinsic function got some semantic tweaks.
+
+There are new intrinsic functions for trigonometric functions in
+units of degrees and half-circles.
+GNU Fortran already supports the forms that use degree units.
+These should call into math library implementations that are
+specialized for those units rather than simply multiplying
+arguments or results with conversion factors.
+ * `ACOSD`, `ASIND`, `ATAND`, `ATAN2D`, `COSD`, `SIND`, `TAND`
+ * `ACOSPI`, `ASINPI`, `ATANPI`, `ATAN2PI`, `COSPI`, `SINPI`, `TANPI`
+
+`SELECTED_LOGICAL_KIND` maps a bit size to a kind of `LOGICAL`
+
+There are two new character utility intrinsic
+functions whose implementations have very low priority: `SPLIT` and `TOKENIZE`.
+`TOKENIZE` requires memory allocation to return its results,
+and could and should have been implemented once in some Fortran utility
+library for those who need a slow tokenization facility rather than
+requiring implementations in each vendor's runtime support library with
+all the extra cost and compatibilty risk that entails.
+
+`SPLIT` is worse -- not only could it, like `TOKENIZE`,
+have been supplied by a Fortran utility library rather than being
+added to the standard, it's redundant;
+it provides nothing that cannot be already accomplished by
+composing today's `SCAN` intrinsic function with substring indexing:
+
+```
+module m
+  interface split
+    module procedure :: split
+  end interface
+  !instantiate for all possible ck/ik/lk combinations
+  integer, parameter :: ck = kind(''), ik = kind(0), lk = kind(.true.)
+ contains
+  simple elemental subroutine split(string, set, pos, back)
+    character(*, kind=ck), intent(in) :: string, set
+    integer(kind=ik), intent(in out) :: pos
+    logical(kind=lk), intent(in), optional :: back
+    if (present(back)) then
+      if (back) then
+        pos = scan(string(:pos-1), set, .true.)
+        return
+      end if
+    end if
+    npos = scan(string(pos+1:), set)
+    pos = merge(pos + npos, len(string) + 1, npos /= 0)
+  end
+end
+```
+
+(The code above isn't a proposed implementation for `SPLIT`, just a
+demonstration of how programs could use `SCAN` to accomplish the same
+results today.)
+
+## Source limitations
+
+Fortran 202X raises the maximum number of characters per free form
+source line and the maximum total number of characters per statement.
+Both of these have always been unlimited in this compiler (or
+limited only by available memory, to be more accurate.)
+
+## More BOZ usage opportunities
+
+BOZ literal constants (binary, octal, and hexadecimal constants,
+also known as "typeless" values) have more conforming usage in the
+new standard in contexts where the type is unambiguously known.
+They may now appear as initializers, as right-hand sides of intrinsic
+assignments to integer and real variables, in explicitly typed
+array constructors, and in the definitions of enumerations.
+
+## Citation updates
+
+The source base contains hundreds of references to the subclauses,
+requirements, and constraints of the Fortran 2018 standard, mostly in code comments.
+These will need to be mapped to their Fortran 202X counterparts once the
+new standard is published, as the Fortran committee does not provide a
+means for citing these items by names that are fixed over time like the
+C++ committee does.
+If we had access to the LaTeX sources of the standard, we could generate
+a mapping table and automate this update.