[docs] LLVM Security Group and Process

[jfb]

Summary

See the corresponding RFC on llvm-dev for a discussion of this proposal.

On this review we're looking for the following feedback: going into specific details, what do you think should be done differently, and what do you think is exactly right in the draft proposal?

[efriedma]

We should explicitly state that patches to LLVM sent to the group are subject to the standard LLVM developer policy/license. This is important so members of the security group can use any patches.

We should prominently state that all messages and attachments will be publicly disclosed after any embargo expires. This is important so issue reporters don't send code under NDAs/etc.

[kcc]

llvm/docs/Security.rst

180

crbug.org has been working well for us e.g. for oss-fuzz or for one-off cases like
https://bugs.chromium.org/p/chromium/issues/detail?id=994957
https://bugs.chromium.org/p/chromium/issues/detail?id=606626

GitHub's security advisories are very recent and unclear if the workflow is polished.
E.g. I can't seem to add comments to the advisory once it's public.
I didn't check if these advisories have an API (they should).

Yet, I think we should consider GitHub as the primary candidate because this is where LLVM is and where the majority of OSS people are.
We may need to ask GitHub to implement missing features, if any.

[christof]

I've not read this in detail or followed the list, but I wanted to add that I believe it's important that we have some form of public acknowledgement for the people that have reported security vulnerabilities in there as well.

[jfb]

• Address feedback

[jfb]

In D70326#1750434, @efriedma wrote:

We should explicitly state that patches to LLVM sent to the group are subject to the standard LLVM developer policy/license. This is important so members of the security group can use any patches.

We should prominently state that all messages and attachments will be publicly disclosed after any embargo expires. This is important so issue reporters don't send code under NDAs/etc.

I'm not aware of projects pointing out their contribution policy in a different manner for security patches. Certainly we want the contributor policy to be prominent, for example if we use GitHub we can add a CONTRIBUTING.md file to do this. I'm just not sure I understand how it should be different for the purpose of security issues.

[jfb]

In D70326#1772570, @christof wrote:

I've not read this in detail or followed the list, but I wanted to add that I believe it's important that we have some form of public acknowledgement for the people that have reported security vulnerabilities in there as well.

CVEs have that property. Folks on the list are worried about "security theater", so I don't think I want to maintain a public leaderboard.

[theraven]

We (Microsoft) are interested in participating in this process.

I have one concern, which is that most of the security issues arising from LLVM are not necessarily security issues in LLVM itself. For example, a miscompilation that breaks invariants that a sandboxing technique depends on will appear to most LLVM developers as a simple miscompilation and may be a security issue only for downstream consumers. Presumably this group should be involved if the issue may apply to multiple downstream consumers?

Where do things like the null-check elision that caught Linux fall? This was a Linux dependence on undefined behaviour that caused compilers to emit code that introduced security vulnerabilities into Linux. Is that in scope for this group, or would we regard that as a Linux vulnerability independent of LLVM? What about, for example, a change to if-conversion that introduces branches in C code believed to be constant time and introduces vulnerabilities in crypto libraries, is that in scope?

I don't think that we can characterize security issues based on where in the code they exist, but rather based on the kinds of behaviour that they trigger and we need to provide very clear advice on what that should be.

[jfb]

In D70326#1809735, @theraven wrote:

We (Microsoft) are interested in participating in this process.

Thanks David.

I have one concern, which is that most of the security issues arising from LLVM are not necessarily security issues in LLVM itself. For example, a miscompilation that breaks invariants that a sandboxing technique depends on will appear to most LLVM developers as a simple miscompilation and may be a security issue only for downstream consumers. Presumably this group should be involved if the issue may apply to multiple downstream consumers?

Where do things like the null-check elision that caught Linux fall? This was a Linux dependence on undefined behaviour that caused compilers to emit code that introduced security vulnerabilities into Linux. Is that in scope for this group, or would we regard that as a Linux vulnerability independent of LLVM? What about, for example, a change to if-conversion that introduces branches in C code believed to be constant time and introduces vulnerabilities in crypto libraries, is that in scope?

I don't think that we can characterize security issues based on where in the code they exist, but rather based on the kinds of behaviour that they trigger and we need to provide very clear advice on what that should be.

I absolutely agree! This complexity is why I'm not trying to decide what's in / out right now, and would rather have that process occur as follow-up RFCs (with updates to this document explaining what's in / out and why). Folks were expressing similar worries on the mailing list.

[aadg]

llvm/docs/Security.rst

25

I understand we have to solve a chicken & egg problem here to get the group started ; I think we should rather say that a call for application to the initial security group should be made, and the board will pick 10 candidates amongst the applications. The board can not possibly know everyone in the community, and to be effective, this group needs volunteers, not people who have been volunteered.

10 seems like a big number of people for an initial group --- given the number of people who expressed interest in the forming of this group, so what should we do if there are less than 10 volunteers ?

The initial task for this group will probably be to finish fleshing up this proposal.

[Shayne]

In D70326#1810421, @jfb wrote:

In D70326#1809735, @theraven wrote:

We (Microsoft) are interested in participating in this process.

Thanks David.

I have one concern, which is that most of the security issues arising from LLVM are not necessarily security issues in LLVM itself. For example, a miscompilation that breaks invariants that a sandboxing technique depends on will appear to most LLVM developers as a simple miscompilation and may be a security issue only for downstream consumers. Presumably this group should be involved if the issue may apply to multiple downstream consumers?

Where do things like the null-check elision that caught Linux fall? This was a Linux dependence on undefined behaviour that caused compilers to emit code that introduced security vulnerabilities into Linux. Is that in scope for this group, or would we regard that as a Linux vulnerability independent of LLVM? What about, for example, a change to if-conversion that introduces branches in C code believed to be constant time and introduces vulnerabilities in crypto libraries, is that in scope?

I don't think that we can characterize security issues based on where in the code they exist, but rather based on the kinds of behaviour that they trigger and we need to provide very clear advice on what that should be.

I absolutely agree! This complexity is why I'm not trying to decide what's in / out right now, and would rather have that process occur as follow-up RFCs (with updates to this document explaining what's in / out and why). Folks were expressing similar worries on the mailing list.

Hello. As David mentioned, we are interested in being involved and I'll be the contact from Microsoft. We have quite a few teams using LLVM to compile parts of their products, and it's going to be important to figure out how to ensure security. As David said, this often will not mean changes to LLVM, but rebuilding the project source with new security fixes/features. Thinking about a previous security vulnerability, Spectre, if this group would have handled it, we could learn about LLVM features to apply to our projects prior to disclosure.

[rogerjarrett]

We (MathWorks) are interested in participating in this process.

I would be the contact from MathWorks. We ship LLVM as part of MATLAB and Simulink; these products are released twice a year.

The proposed document looks great. My only minor suggestion is that the LLVM Security Group have an odd number of members to limit the chance of tie vote.
I agree with theraven and jfb that analysis of security issues should be based on the kinds of behavior that they trigger.

[aadg]

This comment has been deleted.

[aadg]

Considering that:

• this topic and associated review has been open for quite some time already,
• comments have been addressed when possible,
• the LLVM Foundation board believes it is important for our community to have some security process in place, and this proposal is a step in the right direction,
• keeping this stalled will not bring any benefit to the LLVM community (especially since face-to-face discussions at EuroLLVM'20 could not take place, and no one can be sure if the US conference will enable more face to face discussions),

I am approving this on the LLVM Foundation board behalf so that the forming of a security group and the finalization of this process can start, with the community and especially those who have expressed interest in this.

Thanks for the effort in pushing this !

[jfb]

This has received good feedback, I'll therefore commit it in the next few days.

[rengolin]

llvm/docs/Security.rst

25

I agree on both points. We shouldn't burden the foundation with it nor we should restrict the number of members to a fixed size.

52

Redundant wording. Perhaps sub-bullet points?

112

What if the group doesn't have a member from an affected vendor? How do we handle external vendor/country embargo?

[psmith]

llvm/docs/Security.rst

46

There are many vendors that build products from LLVM, and would like to be informed about vulnerabilities, but they may not be able to provide a security expert for the group. We may be at risk of putting off smaller vendors from putting names forward that largely want to be informed but may not be able to contribute fixes.

I don't think this needs changing in the text though. We'll have to see how it goes.

153

Is it worth documenting what happens when the decision that the issue is not security-related? For example update "What is a security issue?" if necessary.

We have time limits and a place for communicating fixes. How and where do we communicate a non-security issue? For example is there a LLVM-DEV post?

I'm sure that there will be some decisions that will need revisiting due to community feedback or further information. I don't think that there needs to be a formal appeals procedure, I think that if the arguments are persuasive the committee can change their mind.

[theraven]

llvm/docs/Security.rst

46

I think that's a great point. We may want to have a two-ring approach, with a group that will coordinate the response and patch, and a wider distribution group that has access to the embargoed patch so that they can do package builds and coordinated releases. My concern over this approach is that it's much lower overhead to be in the second group and so the incentive is to only be in the second group, where you benefit from the process but don't contribute.

This also needs to be balanced with the fact that a leak of an embargoed patch is more likely the more people are exposed to it (for example, OpenBSD leaked the fix for the KRACK WPA2 attack before the embargo, which put everyone else at risk and got the project banned from access to embargoed fixes for a few things).

From the project's perspective, what is the benefit of having these small vendors participating? For the vendor to benefit, they must have a process for handling embargoed fixes and doing coordinated releases. It seems quite unlikely that such a vendor would not have someone who can help our at least in coordinating the response, if not in assessing the security.

[jfb]

Address more comments, add names.

llvm/docs/Security.rst

25

The board's expressed preferred direction is to start with those who have self-identified in the RFC discussion. I'll go with that.

46

Agreed that this is a valid concern. I would like to figure it out as we move forward, not in the first steps of setting up our process.

52

Using sub-bullets makes it unclear what is "and" and what isn't.

112

It won't be perfect at the start, but it's already way more imperfect right now. We should do outreach (such as on this review and the RFC), and over the first few months I think we'll be in a position where what you ask isn't an issue anymore.

153

An issue that isn't deemed part of the security surface area is opened to the public as part of the embargo. In most cases, we'd decide that there's no embargo.

That being said, we can keep an embargo if, for example, it's not part of LLVM's security surface area but is for some non-LLVM project. Say: we use a 3rd party library in a non-secure manner, we're told of an issue, we say "not in out threat model", but other open-source projects have it in their thread model. In such a case we don't want to lift embargo, because it would affect the non-LLVM project.

Ultimately, we'll also do a periodic report where those issues show up.

180

That's been my thinking as well, and one of the first follow-ups I'd like to come after this initial commit.

[jfb]

I believe this is now ready to go, with more to do afterwards.

[jfb]

llvm/docs/Security.rst

41

Tagging @dim @emaste @kristof.beyls @mattdr @ojhunt @probinson @peter.smith @reames @Shayne, they're the folks who have Phabricator accounts from this list.

[jkorous]

llvm/docs/Security.rst

204

We should probably include tools that need to be run with elevated privileges of some sort. For example lldb getting root.

[jfb]

llvm/docs/Security.rst

204

We'd need LLDB maintainers signing up to doing this maintenance. Not that we can't / shouldn't, but that we ought to consider these one at a time, with proper support.

[MaskRay]

Hi, your git commit contains extra Phabricator tags. You can drop Reviewers: Subscribers: Tags: and the text Summary: from the git commit with the following script:

arcfilter () {
        arc amend
        git log -1 --pretty=%B | awk '/Reviewers:|Subscribers:/{p=1} /Reviewed By:|Differential Revision:/{p=0} !p && !/^Summary:$/ {sub(/^Summary: /,"");print}' | git commit --amend --date=now -F -
}

Reviewed By: is considered important by some people. Please keep the tag. (--date=now is my personal preference (author dates are usually not useful. Using committer dates can make log almost monotonic in time))

llvm/utils/git/pre-push.py can validate the message does not include unneeded tags.

[hfinkel]

llvm/docs/Security.rst

177

I recommend that part of this process, presumably at the end, be directed at fulfilling goal #6 above ("Strive to improve security over time, for example by adding additional testing, fuzzing, and hardening after fixing issues."). Maybe something along the lines of: LLVM bug reports will be filed against fuzz testers and/or other components to detail gaps in testing coverage that seem likely to prevent similar cases from arising in the future.

llvm/docs/Contributing.rst

 LLVM from source as described in :doc:`GettingStarted` and
 and use the built binaries to reproduce the failure described in the bug. Use
 a debug build (`-DCMAKE_BUILD_TYPE=Debug`) or a build with assertions
 (`-DLLVM_ENABLE_ASSERTIONS=On`, enabled for Debug builds).
 
 Reporting a Security Issue
 --------------------------
 
-There is a separate process to submit security-related bugs, see :ref:`How to report a security issue?`.
+There is a separate process to submit security-related bugs, see :ref:`report-security-issue`.
 
 Bigger Pieces of Work
 ---------------------
 In case you are interested in taking on a bigger piece of work, a list of
 interesting projects is maintained at the `LLVM's Open Projects page`_. In case
 you are interested in working on any of these projects, please send a mail to
 the `LLVM Developer's mailing list`_, so that we know the project is being
 worked on.

llvm/docs/HowToSubmitABug.rst

 ================================
 How to submit an LLVM bug report
 ================================
 
 Introduction - Got bugs?
 ========================
 
 
 If you're working with LLVM and run into a bug, we definitely want to know
 about it.  This document describes what you can do to increase the odds of
 getting it fixed quickly.
 
-If you believe that the bug is security related, please follow :ref:`How to report a security issue?`.
+🔒 If you believe that the bug is security related, please follow :ref:`report-security-issue`. 🔒
 
 Basically you have to do two things at a minimum.  First, decide whether
 the bug `crashes the compiler`_ (or an LLVM pass), or if the
 compiler is `miscompiling`_ the program (i.e., the
 compiler successfully produces an executable, but it doesn't run right).
 Based on what type of bug it is, follow the instructions in the linked
 section to narrow down the bug so that the person who fixes it will be able
 to find the problem more easily.

llvm/docs/Security.rst

 *Note*: these goals ensure timely action, provide disclosure timing when issues are reported, and respect vendors' / packagers' / users' constraints.
 
 The LLVM Security Group is private. It is composed of trusted LLVM contributors. Its discussions remain within the Security Group (plus issue reporter and key experts) while an issue is being investigated. After an issue becomes public, the entirety of the group’s discussions pertaining to that issue also become public.
 
 
 Group Composition
 =================
 
-Initial group
--------------
+Security Group Members
+----------------------
 
-The initial security group will start small and grow following the process established below. The LLVM Board will pick 10 community members. These members shall represent a wide cross-section of the community, and meet the criteria for inclusion below.
+The members of the group represent a wide cross-section of the community, and meet the criteria for inclusion below.
 
-*FUTURE*: where we maintain a list of current Security Group members can be decided later.
+* Akila Srinivasan (Apple)
+* Dimitry Andric (invidual; FreeBSD)
+* Ed Maste (individual; FreeBSD)
+* JF Bastien (Apple)
+* Josh Eads (Sony)
+* Kristof Beyls (ARM)
+* Matthew Riley (Google)
+* Oliver Hunt (Apple)
+* Paul Robinson (Sony)
+* Peter Smith (ARM)
+* Philip Reames (Azul Systems Inc)
+* Pietro Albini (individual; Rust)
+* Serge Guelton (RedHat)
+* Shayne Hiet-Block (Microsoft)
+* Steve Klabnik (Oxide Computer Company; Rust)
 
 Criteria
 --------
 
 * Nominees for LLVM Security Group membership should fall in one of these groups:
 
   - Individual contributors:
 
     + Specializes in fixing compiler-based security related issues or often participates in their exploration and resolution.
     + Has a track record of finding security vulnerabilities and responsible disclosure of those vulnerabilities.
     + Is a compiler expert who has specific interests in knowing about, resolving, and preventing future security vulnerabilities.
     + Has actively contributed non-trivial code to the LLVM project in the last year.
 
   - Researchers:
 
     + Has a track record of finding security vulnerabilities and responsible disclosure of those vulnerabilities.
     + Is a compiler expert who has specific interests in knowing about, resolving, and preventing future security vulnerabilities.
 
   - Vendor contacts:

 
 
 Privileges and Responsibilities of LLVM Security Group Members
 ==============================================================
 
 Access
 ------
 
 LLVM Security Group members will be subscribed to a private `Discussion Medium`_ (*FUTURE*: see section below). It will be used for technical discussions of security issues, as well as process discussions about matters such as disclosure timelines and group membership. Members have access to all security issues.
 
 Confidentiality
 ---------------
 
 Members of the LLVM Security Group will be expected to treat LLVM security issue information shared with the group as confidential until publicly disclosed:
 
 * Members should not disclose security issue information to non-members unless both members are employed by the same vendor of a LLVM based product, in which case information can be shared within that organization on a need-to-know basis and handled as confidential information normally is within that organization.
 * If the LLVM Security Group agrees, designated members may share issues with vendors of non-LLVM based products if their product suffers from the same issue. The non-LLVM vendor should be asked to respect the issue’s embargo date, and to not share the information beyond the need-to-know people within their organization.

 
 The medium used to host LLVM Security Group discussions is security-sensitive. It should therefore run on infrastructure which can meet our security expectations.
 
 This is where all security discussions occur:
 
 * File security issues.
 * Nominate new members.
 * Propose member removal.
 * Suggest policy changes.
 * Discuss security improvements to LLVM.
 
 
 When a new issue is filed, a template is provided to help issue reporters provide all relevant information.
 
 
 Process
 =======
 
 The following process occurs on the discussion medium for each reported issue:
 
 * A security issue reporter (not necessarily an LLVM contributor) reports an issue.
 * Within two business days, a member of the Security Group is put in charge of driving the issue to an acceptable resolution. This champion doesn’t need to be the same person for each issue. This person can self-nominate.
 * Members of the Security Group discuss in which circumstances (if any) an issue is relevant to security, and determine if it is a security issue.
 * Negotiate an embargo date for public disclosure, with a default minimum time limit of ninety days.
 * Security Group members can recommend that key experts be pulled in to specific issue discussions. The key expert can be pulled in unless there are objections from other Security Group members.
 * Patches are written and reviewed.
 * Backporting security patches from recent versions to old versions cannot always work. It is up to the Security Group to decide if such backporting should be done, and how far back.
 * The Security Group figures out how the LLVM project’s own releases, as well as individual vendors’ releases, can be timed to patch the issue simultaneously.
 * Embargo date can be delayed or pulled forward at the Security Group’s discretion.
 * The issue champion obtains a CVE entry from MITRE_.
 * Once the embargo expires, the patch is posted publicly according to LLVM’s usual code review process.
 * All security issues (as well as nomination / removal discussions) become public within approximately fourteen weeks of the fix landing in the LLVM repository. Precautions should be taken to avoid disclosing particularly sensitive data included in the report (e.g. username and password pairs).
 
 
 Changes to the Policy
 =====================
 
 The LLVM Security Policy may be changed by majority vote of the LLVM Security Group. Such changes also need to be approved by the LLVM Board.
 
 
 What is considered a security issue?
 ====================================
 
 *FUTURE*: this section will be expanded once the Security Group is formed, and it agrees on an initial security surface area.
 
 The LLVM Project has a significant amount of code, and not all of it is considered security-sensitive. This is particularly true because LLVM is used in a wide variety of circumstances: there are different threat models, untrusted inputs differ, and the environment LLVM runs in is varied. Therefore, what the LLVM Project considers a security issue is what its members have signed up to maintain securely.
 
 As this security process matures, members of the LLVM community can propose that a part of the codebase be designated as security-sensitive (or no longer security-sensitive). This requires a rationale, and buy-in from the LLVM community as for any RFC. In some cases, parts of the codebase could be handled as security-sensitive but need significant work to get to the stage where that's manageable. The LLVM community will need to decide whether it wants to invest in making these parts of the code secure-able, and maintain these security properties over time. In all cases the LLVM Security Group should be consulted, since they'll be responding to security issues filed against these parts of the codebase.
 
 If you're not sure whether an issue is in-scope for this security process or not, err towards assuming that it is. The Security Group might agree or disagree and will explain its rationale in the report, as well as  update this document through the above process.
 
 The security-sensitive parts of the LLVM Project currently are:
 
 * None (this process is new, the list hasn't been populated yet)
+* *FUTURE*: this section will be expanded.
+
+The parts of the LLVM Project which are currently treated as non-security sensitive are:
+
+* Language front-ends, such as clang, for which a malicious input file can cause undesirable behavior. For example, a maliciously-crafter C or Rust source file can cause arbitrary code to execute in LLVM. These parts of LLVM haven't been hardened, and compiling untrusted code usually also includes running utilities such as `make` which can more readily perform malicious things.
+* *FUTURE*: this section will be expanded.
 
+.. _report-security-issue:
 
 How to report a security issue?
 ===============================
 
 *FUTURE*: this section will be expanded once we’ve figured out other details above.
 
 Not everyone who wants to report a security issue will be familiar with LLVM, its community, and processes. Therefore, this needs to be easy to find on the LLVM website, and set clear expectations to issue reporters.
 
 
 
 .. _CVE process: https://cve.mitre.org
 .. _chromium issue tracker: https://crbug.com
 .. _GitHub security: https://help.github.com/en/articles/about-maintainer-security-advisories
 .. _MITRE: https://cve.mitre.org

llvm/docs/index.rst

 * :doc:`GettingInvolved`
 * :ref:`development-process`
 * :ref:`mailing-lists`
 * :ref:`meetups-social-events`
 * :ref:`community-proposals`
 
   Reporting a security issue
 
-* :ref:`How to report a security issue?`
+* :ref:`report-security-issue`
 
 Indices and tables
 ==================
 
 * :ref:`genindex`
 * :ref:`search`