[libc++][ci] Detect not committed generated files.

[Mordante]

Summary

The Generated output CI job only tests for modified files. This job
should also fail the generated output contains new files.

It would be possible to test modified and untracked files in one
execution of git ls-files. However the diff is stored as an artifact
so the execution of git diff would still be required.

Discussion: Would it be better to do git ls-files -om and remove the
excution of
! grep -q '^--- a' ${BUILD_DIR}/generated_output.patch || false ?
(Obviously then the name generated_output.untracked should change to
something like generated_output.status)

[Mordante]

Delete a generated file.
This should cause the build to fail.

[Mordante]

The build passed unexpectedly.
Use the || false method used with grep.

[Mordante]

The last build failed as expected.
Readded the missing file, but it contains manual changes.

[Mordante]

Last build failed as expected.
Remove all manual modifications to the generated files.

[Mordante]

The last build passed as expected.
Restore the removed CI pipelines.

[ldionne]

libcxx/utils/ci/run-buildbot

171–181

Would something like this work:

if ! git ls-files -om; then
    echo "It looks like not all the generator scripts were run, did you forget to build the libcxx-generate-files target?"
    false
fi

[Mordante]

libcxx/utils/ci/run-buildbot

171–181

That doesn't work since the return value of git ls-files -om doesn't indicate a status, I earlier.
I'll rework the patch to generate ${BUILD_DIR}/generated_output.status,
then use if [ -s ${BUILD_DIR}/generated_output.status ]; then to issue a diagnostic.

[Mordante]

Use an alternative approach as @ldionne suggested. This should result in better diagnostics.
Disable the unneeded CI steps for testing purposes.

[Mordante]

Now enable the proper test instead of the inverted test.

[Mordante]

The last build passed as expected.
Remove a generated file to make it fail.

[Mordante]

The last build failed as expected.
Readd the file, but with manual modifications.

[Mordante]

The last build failed as expected.
Undo the changes to the file and the build should pass again.

[Mordante]

The last build passed as expected.
Restore the removed CI steps.

[Mordante]

@ldionne I addressed your comment and tested it. It's ready for review again.

libcxx/utils/ci/buildkite-pipeline.yml

retry:
           limit: 2
     soft_fail:
         - exit_status: 1
 
   - label: "Generated output"
     command: "libcxx/utils/ci/run-buildbot check-generated-output"
     artifact_paths:
       - "**/generated_output.patch"
+      - "**/generated_output.status"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
 

libcxx/utils/ci/run-buildbot

 check-generated-output)
     # `! foo` doesn't work properly with `set -e`, use `! foo || false` instead.
     # https://stackoverflow.com/questions/57681955/set-e-does-not-respect-logical-not
     clean
     generate-cmake
 
     # Reject patches that forgot to re-run the generator scripts.
     echo "+++ Making sure the generator scripts were run"
     ${NINJA} -vC "${BUILD_DIR}" libcxx-generate-files
     git diff | tee ${BUILD_DIR}/generated_output.patch
+    git ls-files -o --exclude-standard | tee ${BUILD_DIR}/generated_output.status
     ! grep -q '^--- a' ${BUILD_DIR}/generated_output.patch || false
+    if [ -s ${BUILD_DIR}/generated_output.status ]; then
+        echo "It looks like not all the generator scripts were run,"
+        echo "did you forget to build the libcxx-generate-files target?"
+        echo "Did you add all new files it generated?"
+        false
+    fi
 
     # Reject patches that introduce non-ASCII characters or hard tabs.
     # Depends on LC_COLLATE set at the top of this script.
     ! grep -rn '[^ -~]' libcxx/include/ || false
 
     # Reject patches that introduce dependency cycles in the headers.
     python3 libcxx/utils/graph_header_deps.py >/dev/null
 ;;
 generic-cxx03)