[llvm-objcopy][NFC] replace class Buffer/MemBuffer/FileBuffer with streams.

[avl]

Summary

During D88827 it was requested to remove the local implementation
of Memory/File Buffers:

// TODO: refactor the buffer classes in LLVM to enable us to use them here
// directly.

This patch uses raw_ostream instead of Buffers. Generally, using streams
could allow us to reduce memory usages. No need to load all data into the
memory - the data could be streamed through a smaller buffer.
Thus, this patch uses raw_ostream as an interface for output data:

Error executeObjcopyOnBinary(CopyConfig &Config,
                             object::Binary &In,
                             raw_ostream &Out);

Note 1. This patch does not change the implementation of Writers
so that data would be directly stored into raw_ostream.
This is assumed to be done later.

Note 2. It would be better if Writers would be implemented in a such way
that data could be streamed without seeking/updating. If that would be
inconvenient then raw_ostream could be replaced with raw_pwrite_stream
to have a possibility to seek back and update file headers.
This is assumed to be done later if necessary.

Note 3. Current FileOutputBuffer allows using a memory-mapped file.
The raw_fd_ostream (which could be used if data should be stored in the file)
does not allow us to use a memory-mapped file. Memory map functionality
could be implemented for new kind of stream raw_mmap_stream.
This is assumed to be done later if necessary.

[avl]

ping.

[jhenderson]

I've skimmed this briefly, and it looks generally okay, but I've got too much else going on to give this a proper review. I'd prefer it if somebody else could take a look.

llvm/tools/llvm-objcopy/Util.h

10

Here and below.

24–27

No need for else after return.

44–45

You don't need this.

[grimar]

This also looks generally OK to me (after fixing style issues mentioned by James). Few more minor nits from me inline.

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

100

I'd not use auto here. You can use an actual type instead.

104

The same, probably.

136

Do you need this helper? I think generally the direction is to get rid of error codes.
I am not sure if this method is usefull? You can just craft a better error message using createStringError in place.

177

Perhaps, link->rename?

[avl]

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

100

these findBuildID, makeStringError, linkToBuildIdDir functions are not written from the scratch.
They were moved without modifications from ELF/ELFObjcopy.cpp.
I would prefer to not change them, since it would be separate refactoring not related to the current one.

Would it be OK, to make those changes as a separate patch?

[grimar]

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

100

I see. Sorry, I haven't realized it. Sure, then lets keeps them as is for now.

[avl]

addressed comments.

[avl]

ping.

update for Note 3. It was considered to be not useful to implement reserve() method as fixed-size memory-mapped file(D91693). Instead it was suggested to implement resizable raw_mmap_ostream.
Thus, if we need to have memory mapped file for objcopy then we have this alternative - implement resizable raw_mmap_ostream.

[jhenderson]

In D91028#2410699, @avl wrote:

ping.

update for Note 3. It was considered to be not useful to implement reserve() method as fixed-size memory-mapped file(D91693). Instead it was suggested to implement resizable raw_mmap_ostream.
Thus, if we need to have memory mapped file for objcopy then we have this alternative - implement resizable raw_mmap_ostream.

FYI, I'm holding off reviewing this patch further (and I'd recommend others do the same personally) until we've sorted out D91693, as I think that will continue to have an impact on this patch.

[avl]

@rupprecht @alexshap @grimar @dblaikie

Colleagues, so what is your opinion on this refactoring? Is it OK to use streams as output data for llvm-objcopy?

[dblaikie]

Hopefully some other folks with more investment in dsymutil chime in here, but at least:

This patch uses raw_ostream instead of Buffers. Generally, using streams could allow us to reduce memory usages. No need to load all data into the memory - the data could be streamed through a smaller buffer.

Aren't many MemoryBuffers backed by memory mapped files, which aren't necessarily causing real memory usage? Could that approach be used here?

That said, pretty sure LLVM's integrated assembler uses a pwrite stream to write out its object files, so if we can/if this patch does align with that method of output that seems OK.

I'm confused by the tenses in some of the notes in the patch description:

Note 2. It would be better if Writers would be implemented in a such way that data could be streamed without seeking/updating. If that would be inconvenient then raw_ostream could be replaced with raw_pwrite_stream to have a possibility to seek back and update file headers. This is assumed to be done later if necessary.

So this patch as proposed is not using a pwrite stream, it seems? "if Writers could be implemented in such a way that data could be streamed without seeking/updating" - how does that work today, without the patch, and how does the patch address that need to seek? I don't believe it's possible to write an ELF file without either knowing at least the sizes of all the sections up-front (you might be able to know these sizes without actually buffering the entire contents of the file) or seeking. Because the header needs to either contain an offset to the section table stored at the end of the file, where it knows the offsets/lengths of all the sections, or you write the section table at the start (so the header knows the offset up-front) and the table then needs to know the offsets of all the sections up-front.

[avl]

Aren't many MemoryBuffers backed by memory mapped files, which aren't necessarily causing real memory usage? Could that approach be used here?

That is right. Many MemoryBuffers are backed by memory-mapped files, which aren't necessarily causing real memory usage.
But if we could not use a memory-mapped file then we do not have another way than loading the entire output file into the memory.
Using streams allows us to have another way than loading the entire file into the memory(for the cases when we could not use memory-mapped files).
f.e. there already exist the following code in the codebase:

Expected<std::unique_ptr<MemoryBuffer>>
object::writeUniversalBinaryToBuffer(ArrayRef<Slice> Slices) {
  SmallVector<char, 0> Buffer;
  raw_svector_ostream Out(Buffer);

  if (Error E = writeUniversalBinaryToStream(Slices, Out))  <<< entire output file is loaded into the memory
    return std::move(E);
    
  return std::make_unique<SmallVectorMemoryBuffer>(std::move(Buffer));
}    
......    
  Expected<std::unique_ptr<MemoryBuffer>> B =
      writeUniversalBinaryToBuffer(Slices);
  if (!B)
    return B.takeError();
  if (Error E = Out.allocate((*B)->getBufferSize()))
    return E;
  memcpy(Out.getBufferStart(), (*B)->getBufferStart(), (*B)->getBufferSize());  
  ^^^^^^^^^
  data is copied from memory into the output buffer

We could not use the memory-mapped file for writeUniversalBinaryToStream() since we do not know the final size.
Opposite, streams do not require knowing the final size so writeUniversalBinaryToStream() might write to the output
directly, without intermediate buffers.

That said, pretty sure LLVM's integrated assembler uses a pwrite stream to write out its object files, so if we can/if this patch does align with that method of output that seems OK.

That is one alternative which I suggest. We could use raw_pwrite_stream. If everybody agrees that it is correct choice I could update the patch accordingly.

But, initially, I suggested trying to use a, probably, better alternative - using raw_ostream.

So this patch as proposed is not using a pwrite stream, it seems?

yes. the patch in the current form suggests using raw_ostream. The reason for this is that raw_ostream is more general and has fewer restrictions
(it does not require to have seek/update functionality). It is good for the library to have a more general interface.

"if Writers could be implemented in such a way that data could be streamed without seeking/updating" - how does that work today, without the patch, and how does the patch address that need to seek?

without the patch, the data is already pre-allocated by external in-memory/mmap buffer:

llvm/tools/llvm-objcopy/ELF/Object.cpp

template <class ELFT> Error ELFWriter<ELFT>::finalize() {
...
  if (Error E = Buf.allocate(totalSize())) <<<<<<<<<<<<<<<<<<<<<<<
  return E;
...

So it is possible to seek into any place and update.

with this patch, an additional pre-allocated internal memory buffer is used. the data still could be randomly accessed.
When the writing is done then the whole buffer is written into the stream. The cost of that solution - an additional buffer.

Buf = WritableMemoryBuffer::getNewMemBuffer(totalSize());
...
// write ELF output to buf.
...
// TODO: Implement direct writing to the output stream
// TODO: (without intermediate memory buffer Buf).
Out.write(Buf->getBufferStart(), Buf->getBufferSize());
Out.flush();

I don't believe it's possible to write an ELF file without either knowing at least the sizes of all the sections up-front (you might be able to know these sizes without actually buffering the entire contents of the file) or seeking. Because the header needs to either contain an offset to the section table stored at the end of the file, where it knows the offsets/lengths of all the sections, or you write the section table at the start (so the header knows the offset up-front) and the table then needs to know the offsets of all the sections up-front.

I do not know ELF format/implementation well. But current implementation of ELF writer knows the size of the resulting file _before_ any writing started
(It allocates a buffer of the proper size). Since it knows the total output size, probably, it could pre-calculate offset to the section table for ELF header before writing started?

My initial suggestion is to try to use a more general interface(raw_ostream) and change it into the more specific(raw_pwrite_stream) if it would be inconvenient to use raw_ostream:

1. Implement interfaces using raw_ostream:

Error executeObjcopyOnBinary(CopyConfig &Config,
                             object::Binary &In,
                             raw_ostream &Out);

2. Use additional internal buffers to not to rewrite the writer's implementation.

3. Someone(probably me, but later) changes the implementation of writers(ELF/COFF/MachO/Wasm) to not use internal buffers. So that writers store data into the output stream directly.

4. If all implementations are successful - then leave raw_ostream in interfaces.

5. If some implementations still require to seek/update functionality then change raw_ostream into raw_pwrite_stream:

Error executeObjcopyOnBinary(CopyConfig &Config,
                             object::Binary &In,
                             raw_pwrite_stream &Out);

If it is already clear that we need to use raw_pwrite_stream then it might be OK to use it in this patch from the start.

[avl]

rebased.

[sbc100]

We could not use the memory-mapped file for writeUniversalBinaryToStream() since we do not know the final size.

Is there some fundamental reason why you can't determine the final output size? In lld (wasm-ld at least) we go to some lengths to determine the final size before writing, basically for this reason. IIUC using a memory mapped output file has some big wins.

[avl]

Is there some fundamental reason why you can't determine the final output size? In lld (wasm-ld at least) we go to some lengths to determine the final size before writing, basically for this reason.

I do not know whether exists some fundamental reason for not determining output size in the above case.
My comments based on the existing code:

Expected<std::unique_ptr<MemoryBuffer>> B =
    writeUniversalBinaryToBuffer(Slices);
if (!B)
  return B.takeError();
if (Error E = Out.allocate((*B)->getBufferSize()))
  return E;
memcpy(Out.getBufferStart(), (*B)->getBufferStart(), (*B)->getBufferSize());
return Out.commit();

Thus, I assumed that in that case, it is hard to calculate the final size.
Probably, the reason why the data is copied here is just not presenting class "Buffer" in the Support or Object libraries(So it could not be used inside the writeUniversalBinaryToBuffer()).

In the general case, the calculation of the final size might be the same expensive as the generation of the output file.
So it might be ineffective to pre-calculate the size.
That is why I am trying to suggest an interface that would not require buffer pre-allocation.

IIUC using a memory mapped output file has some big wins.

My suggestion is not about avoiding memory-mapped file usages.
I suggest an interface that allows to select/change the most useful data holder.
f.e. If we need to calculate SHA on the output of llvm-objcopy we would need
to store data in some temporary location and then do calculations:

MemBuffer/FileBuffer Out;
executeObjcopyOnBinary(Config, In, Out);
calculateSha(Out);

With the stream approach we do not need to allocate any temporary data at all:

raw_sha1_ostream Out;  <<< calculates SHA, do not allocate any data.
executeObjcopyOnBinary(Config, In, Out);

How memory-mapped files could be used with streams - (that solution suggested in https://reviews.llvm.org/D91693#2407437):

implement resizable mmap_ostream. mmap_ostream would allocate memory-mapped chunks.  If it needs to store more data it would allocate new chunks. Finally, it closes and concatenates all chunks into a single file.
if implementation uses reserve() method to inform mmap_ostream which size to preallocate then mmap_ostream would have the same performance as current FileOutputBuffer.

That solution allows to:

1. select the most useful data holder.
2. to use size precalculations only when it is cheap.

[avl]

ping.

short summary why that refactoring is useful: It replaces specialized interfaces Buffer/MemBuffer/FileBuffer with more general interface - raw_ostream. It allows easily redefine the type of destination. It also allows to work effectively for implementations which do not pre-allocate destination space.

[dblaikie]

Generally looks OK to me. Though would be good to have @jhenderson, @rupprecht, and maybe @rnk (for the COFF suport) take a look too.

(this is closer to how LLVM emits objects (it uses a raw_pwrite_stream, so it can go back to write the header since it doesn't compute the total layout/length up-front) but a bit further from how lld emits its output (it uses the llvm::FileOutputBuffer which is underneath the Buffer this patch moves away from) - which is a bit awkward, but at least maybe it's less of a 3rd way of doing things... maybe. Wolud be good to try to unify all these things some day)

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

95

Might be easier to review if this code was moved around in a separate preparatory commit. So the code changes would be separate from the code moving.

[jhenderson]

llvm/tools/llvm-objcopy/COFF/Writer.cpp

387–388

You don't need TODO: on this second line. Also, your comment seems to have wrapped far earlier than the 80 character column limit, so you should reflow the comment. clang-format can reflow comments that are too long, as needed, although I don't think it does the inverse (i.e. reflow comments that are shorter than needed), so the easiest thing to do is just delete the new line and run clang-format on the comment.

390

Do we actually need this explicit flush? It seems to me like streams should flush before destruction, so there's no need to explicitly do it here, I'd think.

llvm/tools/llvm-objcopy/ELF/ELFObjcopy.cpp

831

@MaskRay deleted the build ID directory code in another patch (see https://reviews.llvm.org/D96310). We shouldn't be seeing that deletion in this patch too. Please rebase.

llvm/tools/llvm-objcopy/ELF/Object.cpp

2417

Same as above. Do we need the flush?

2550–2553

Same as above re. flush and TODO comments.

2639

Same as above re. flush.

llvm/tools/llvm-objcopy/MachO/MachOObjcopy.cpp

466

I'm not sure if it ever happened, but there was some discussion on the mailing lists about making 0 the default value if the size wasn't specified. Please could you check whether you need this second parameter still.

486

Same comment as above re. flush.

llvm/tools/llvm-objcopy/MachO/MachOWriter.cpp

517

If I'm not mistaken, this and equivalent messages elsewhere will just result in something like the following being printed:

error: 'foo.o': 42

You need to give the full description of the message (i.e. something like "failed to allocate memory buffer of XX bytes").

524–527

Same comment re. todo comments and flush.

llvm/tools/llvm-objcopy/Util.h

19–20

Why is this function now in a new header file? Could it not be declared in one of the existing headers (like llvm-objcopy.h) and defined in the corresponding cpp?

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

95

+1 - this code move is unrelated. Please do it in a separate patch. Some of it might even be deletable after rebasing on D96310).

212–213

What's going on with this comment change? It looks like you've messed it up a bit.

I think what you want is "For regular files (as is the case for deepWriteArchive), FileOutputBuffer::create will return OnDiskBuffer."

302

Same comment as above.

407–410

I don't understand this change? This seems to be doing something quite different to what it was doing before.

llvm/tools/llvm-objcopy/wasm/Writer.cpp

77–80

Same comment as above re. flush and todo.

[avl]

Will address all comments. I have a two questions/comments inside...

llvm/tools/llvm-objcopy/MachO/MachOObjcopy.cpp

466

There is a comment for SmallVector that recommended practice is to not specify default value if there is no well motivated choice. Then it would automatically select one. In this case I think we do not need any stack data. So I specified 0 to avoid stack allocation, which looks correct in this case.

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

407–410

It looks like it does the same. Originally, there were created a file called Config.OutputFilename and ProcessRaw put data in it. In this patch writeToFile helper creates a file called Config.OutputFilename and calls ProcessRaw to store data into it.

[avl]

rebased. addressed comments.

[avl]

@jhenderson Hi James, Is it OK now?

[avl]

@rupprecht @rnk Do you have comments/objections for this refactoring?

[jhenderson]

Sorry, I've been unwell, and not been able to look more at this yet.

Also, one of our downstream users has recently run into a bug which probably impacts users of raw_fd_ostream, when writing from a Windows VM to a MacOS shared directory - they encounter "bad file descriptor" errors, in a manner simialr to https://bugs.llvm.org/show_bug.cgi?id=42623. I haven't filed a bug for this yet, as we're still trying to narrow down what is likely causing the problem, but I'm about 90% certain it is a general bug in raw_fd_ostream in this situation (copying to a memory mapped file as opposed to calling ::write works). At the moment, this bug affects llvm-objcopy when copying archives, but not objects, as the two wirting paths are different. It would be a shame if we introduced the bug into more areas, and I think this change does that. Perhaps we should try fixing that problem first?

[avl]

Sorry, I've been unwell, and not been able to look more at this yet.

Also, one of our downstream users has recently run into a bug which probably impacts users of raw_fd_ostream, when writing from a Windows VM to a MacOS shared directory - they encounter "bad file descriptor" errors, in a manner simialr to https://bugs.llvm.org/show_bug.cgi?id=42623. I haven't filed a bug for this yet, as we're still trying to narrow down what is likely causing the problem, but I'm about 90% certain it is a general bug in raw_fd_ostream in this situation (copying to a memory mapped file as opposed to calling ::write works). At the moment, this bug affects llvm-objcopy when copying archives, but not objects, as the two wirting paths are different. It would be a shame if we introduced the bug into more areas, and I think this change does that. Perhaps we should try fixing that problem first?

I think yes it would be better to fix the problem first. Though we need a reproducer for that problem to not wait forever...

It looks like https://bugs.llvm.org/show_bug.cgi?id=42623 is already resolved in llvm by https://reviews.llvm.org/D81803. (But it has a problem with Windows7, which caused reverting in the rust compiler https://github.com/rust-lang/rust/issues/82677)

Please, let me known if I may help with fixing this.

[jhenderson]

In D91028#2599949, @avl wrote:

Sorry, I've been unwell, and not been able to look more at this yet.

Also, one of our downstream users has recently run into a bug which probably impacts users of raw_fd_ostream, when writing from a Windows VM to a MacOS shared directory - they encounter "bad file descriptor" errors, in a manner simialr to https://bugs.llvm.org/show_bug.cgi?id=42623. I haven't filed a bug for this yet, as we're still trying to narrow down what is likely causing the problem, but I'm about 90% certain it is a general bug in raw_fd_ostream in this situation (copying to a memory mapped file as opposed to calling ::write works). At the moment, this bug affects llvm-objcopy when copying archives, but not objects, as the two wirting paths are different. It would be a shame if we introduced the bug into more areas, and I think this change does that. Perhaps we should try fixing that problem first?

I think yes it would be better to fix the problem first. Though we need a reproducer for that problem to not wait forever...

It looks like https://bugs.llvm.org/show_bug.cgi?id=42623 is already resolved in llvm by https://reviews.llvm.org/D81803. (But it has a problem with Windows7, which caused reverting in the rust compiler https://github.com/rust-lang/rust/issues/82677)

Please, let me known if I may help with fixing this.

Ah, that might be good news. The user is using an LLVM 10 based toolchain, I believe, so wouldn't have this fix in. It's quite possible that when they upgrade to a later LLVM version, they will no longer see the problem. Unfortunately, the user is external to our company, and we don't currently have the setup to confirm the issue either way ourselves. However, by the sounds of it, if somebody does have a Windows 10 VM running using Parallels on a Mac, they should be able to reproduce by doing any generic archive copying with llvm-objcopy or writing using llvm-ar. If anybody on here has such a setup, it would be great if we can get that verification. LLVM 12 meanwhile hasn't yet been released, so I don't think it's practical (yet) for us to ask our user to try to confirm the fix for us.

If nobody does have the ability to reproduce this, I don't think we need to block this review on that, given the likely chance this has been fixed. I'll take a look at this patch again later today, assuming I get a moment.

[jhenderson]

In D91028#2602589, @jhenderson wrote:

In D91028#2599949, @avl wrote:

Sorry, I've been unwell, and not been able to look more at this yet.

Also, one of our downstream users has recently run into a bug which probably impacts users of raw_fd_ostream, when writing from a Windows VM to a MacOS shared directory - they encounter "bad file descriptor" errors, in a manner simialr to https://bugs.llvm.org/show_bug.cgi?id=42623. I haven't filed a bug for this yet, as we're still trying to narrow down what is likely causing the problem, but I'm about 90% certain it is a general bug in raw_fd_ostream in this situation (copying to a memory mapped file as opposed to calling ::write works). At the moment, this bug affects llvm-objcopy when copying archives, but not objects, as the two wirting paths are different. It would be a shame if we introduced the bug into more areas, and I think this change does that. Perhaps we should try fixing that problem first?

I think yes it would be better to fix the problem first. Though we need a reproducer for that problem to not wait forever...

It looks like https://bugs.llvm.org/show_bug.cgi?id=42623 is already resolved in llvm by https://reviews.llvm.org/D81803. (But it has a problem with Windows7, which caused reverting in the rust compiler https://github.com/rust-lang/rust/issues/82677)

Please, let me known if I may help with fixing this.

Ah, that might be good news. The user is using an LLVM 10 based toolchain, I believe, so wouldn't have this fix in. It's quite possible that when they upgrade to a later LLVM version, they will no longer see the problem. Unfortunately, the user is external to our company, and we don't currently have the setup to confirm the issue either way ourselves. However, by the sounds of it, if somebody does have a Windows 10 VM running using Parallels on a Mac, they should be able to reproduce by doing any generic archive copying with llvm-objcopy or writing using llvm-ar. If anybody on here has such a setup, it would be great if we can get that verification. LLVM 12 meanwhile hasn't yet been released, so I don't think it's practical (yet) for us to ask our user to try to confirm the fix for us.

If nobody does have the ability to reproduce this, I don't think we need to block this review on that, given the likely chance this has been fixed. I'll take a look at this patch again later today, assuming I get a moment.

Aside: can https://bugs.llvm.org/show_bug.cgi?id=42623 be resolved now?

[jhenderson]

llvm/tools/llvm-objcopy/COFF/Writer.cpp

387–391

llvm/tools/llvm-objcopy/ELF/Object.cpp

2416

2541

Elsewehere in this file, we don't qualify llvm::errc with the llvm::. I don't think you need it here either.

2543

LLVM coding standards state that errors shouldn't end in '.' - please delete the trailing one here.

2550–2554

2584–2586

As above. Remove the llvm:: qualifier from llvm::errc and delete the trailing full stop from the message.

2638

2687

Please write a more useful error message. This will result in a useless message of something like "error: foo.o: 0". This is the same issue as I already commented on in the mach-o area.

2702–2704

As above. Delete llvm:: and the trailing full stop.

llvm/tools/llvm-objcopy/MachO/MachOWriter.cpp

518–520

As above - no llvm:: or .

524

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

212–213

llvm/tools/llvm-objcopy/wasm/Writer.cpp

62–64

77

[avl]

will address comments. as to checking the "Windows 10 VM running using Parallels on a Mac" configuration - unfortunately, I do not have that hardware/configuration. So I would not be able to check it in the nearest future.

[avl]

adressed comments.

[jhenderson]

LGTM, after the remaining nit is fixed. It might be worth trying to get somebody to confirm the discussed issue has been fixed. Perhaps worth asking on llvm-dev. A simple reproducible is to do something like "llvm-objcopy test.a path/on/shared/drive/test.a".

llvm/tools/llvm-objcopy/ELF/Object.cpp

2704

[avl]

It might be worth trying to get somebody to confirm the discussed issue has been fixed. Perhaps worth asking on llvm-dev. A simple reproducible is to do something like "llvm-objcopy test.a path/on/shared/drive/test.a".

So far I managed to reproduce the problem in a bit different environment(Ubuntu host, Virtual Box VM, Windows guest). Reverting the https://reviews.llvm.org/D81803 patch restores the "LLVM ERROR: IO failure on output stream: bad file descriptor" problem. So I can confirm that https://reviews.llvm.org/D81803 cures the "bad file descriptor" problem. Though it seems has another problem with windows7(as it is already reported in https://reviews.llvm.org/D81803) and it might be reverted.

At the same time, it looks like this patch(https://reviews.llvm.org/D91028) does not change the status quo. The problem from https://reviews.llvm.org/D81803 is with TempFile::keep(). The TempFile::keep() is used by this patch for streams as well as it is used in the current mainline for OnDiskBuffer.commit(). i.e. the same problem arises with the current mainline and with this patch. Thus, I think, the problem with TempFile::keep() does not prevent integrating of this D91028.

[jhenderson]

In D91028#2612219, @avl wrote:

It might be worth trying to get somebody to confirm the discussed issue has been fixed. Perhaps worth asking on llvm-dev. A simple reproducible is to do something like "llvm-objcopy test.a path/on/shared/drive/test.a".

So far I managed to reproduce the problem in a bit different environment(Ubuntu host, Virtual Box VM, Windows guest). Reverting the https://reviews.llvm.org/D81803 patch restores the "LLVM ERROR: IO failure on output stream: bad file descriptor" problem. So I can confirm that https://reviews.llvm.org/D81803 cures the "bad file descriptor" problem. Though it seems has another problem with windows7(as it is already reported in https://reviews.llvm.org/D81803) and it might be reverted.

At the same time, it looks like this patch(https://reviews.llvm.org/D91028) does not change the status quo. The problem from https://reviews.llvm.org/D81803 is with TempFile::keep(). The TempFile::keep() is used by this patch for streams as well as it is used in the current mainline for OnDiskBuffer.commit(). i.e. the same problem arises with the current mainline and with this patch. Thus, I think, the problem with TempFile::keep() does not prevent integrating of this D91028.

The problem our downstream user has run into is seen only when using llvm-objcopy to copy archives, not regular objects, likely because it is only triggered by the writing via raw_fd_ostream, which is not done for regular object files. This change appears to extend the problem to object files too, if I'm not mistaken, hence my concern. However, if the issue is fixed, then it is not an issue either way.

FWIW, I don't think LLVM should be supporting runnning on Windows 7 anymore, since Windows 7 is now over a year past end of life, but that is a separate discussion.

[avl]

The problem our downstream user has run into is seen only when using llvm-objcopy to copy archives, not regular objects, likely because it is only triggered by the writing via raw_fd_ostream, which is not done for regular object files. This change appears to extend the problem to object files too, if I'm not mistaken, hence my concern. However, if the issue is fixed, then it is not an issue either way.

Right. Looking at the fix for D81803 I had impression that only TempFile::keep() is affected. But after some debugging I see that ::write(from unistd.h) function(used by raw_fd_ostream) works incorrectly without the fix. Anyway D81803 seems cured the problem. thanks.

llvm/include/llvm/Object/MachOUniversalWriter.h

if (Rhs.CPUType == MachO::CPU_TYPE_ARM64)
       return true;
     // Sort by alignment to minimize file size
     return Lhs.P2Alignment < Rhs.P2Alignment;
   }
 };
 
 Error writeUniversalBinary(ArrayRef<Slice> Slices, StringRef OutputFileName);
 
-Expected<std::unique_ptr<MemoryBuffer>>
-writeUniversalBinaryToBuffer(ArrayRef<Slice> Slices);
+Error writeUniversalBinaryToStream(ArrayRef<Slice> Slices, raw_ostream &Out);
 
 } // end namespace object
 
 } // end namespace llvm
 
 #endif // LLVM_OBJECT_MACHOUNIVERSALWRITER_H

llvm/lib/Object/MachOUniversalWriter.cpp

for (const auto &S : Slices) {
     FatArch.size = S.getBinary()->getMemoryBufferRef().getBufferSize();
     FatArch.align = S.getP2Alignment();
     Offset += FatArch.size;
     FatArchList.push_back(FatArch);
   }
   return FatArchList;
 }
 
-static Error writeUniversalBinaryToStream(ArrayRef<Slice> Slices,
+Error object::writeUniversalBinaryToStream(ArrayRef<Slice> Slices,
                                            raw_ostream &Out) {
   MachO::fat_header FatHeader;
   FatHeader.magic = MachO::FAT_MAGIC;
   FatHeader.nfat_arch = Slices.size();
 
   Expected<SmallVector<MachO::fat_arch, 2>> FatArchListOrErr =
       buildFatArchList(Slices);
   if (!FatArchListOrErr)
     return FatArchListOrErr.takeError();
Error object::writeUniversalBinary(ArrayRef<Slice> Slices,
   raw_fd_ostream Out(Temp->FD, false);
   if (Error E = writeUniversalBinaryToStream(Slices, Out)) {
     if (Error DiscardError = Temp->discard())
       return joinErrors(std::move(E), std::move(DiscardError));
     return E;
   }
   return Temp->keep(OutputFileName);
 }
-
-Expected<std::unique_ptr<MemoryBuffer>>
-object::writeUniversalBinaryToBuffer(ArrayRef<Slice> Slices) {
-  SmallVector<char, 0> Buffer;
-  raw_svector_ostream Out(Buffer);
-
-  if (Error E = writeUniversalBinaryToStream(Slices, Out))
-    return std::move(E);
-
-  return std::make_unique<SmallVectorMemoryBuffer>(std::move(Buffer));
-}

llvm/tools/llvm-objcopy/Buffer.h

-//===- Buffer.h -------------------------------------------------*- C++ -*-===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-#ifndef LLVM_TOOLS_OBJCOPY_BUFFER_H
-#define LLVM_TOOLS_OBJCOPY_BUFFER_H
-
-#include "llvm/ADT/StringRef.h"
-#include "llvm/Support/FileOutputBuffer.h"
-#include "llvm/Support/MemoryBuffer.h"
-#include <memory>
-
-namespace llvm {
-namespace objcopy {
-
-// The class Buffer abstracts out the common interface of FileOutputBuffer and
-// WritableMemoryBuffer so that the hierarchy of Writers depends on this
-// abstract interface and doesn't depend on a particular implementation.
-// TODO: refactor the buffer classes in LLVM to enable us to use them here
-// directly.
-class Buffer {
-  StringRef Name;
-
-public:
-  virtual ~Buffer();
-  virtual Error allocate(size_t Size) = 0;
-  virtual uint8_t *getBufferStart() = 0;
-  virtual Error commit() = 0;
-
-  explicit Buffer(StringRef Name) : Name(Name) {}
-  StringRef getName() const { return Name; }
-};
-
-class FileBuffer : public Buffer {
-  std::unique_ptr<FileOutputBuffer> Buf;
-  // Indicates that allocate(0) was called, and commit() should create or
-  // truncate a file instead of using a FileOutputBuffer.
-  bool EmptyFile = false;
-
-public:
-  Error allocate(size_t Size) override;
-  uint8_t *getBufferStart() override;
-  Error commit() override;
-
-  explicit FileBuffer(StringRef FileName) : Buffer(FileName) {}
-};
-
-class MemBuffer : public Buffer {
-  std::unique_ptr<WritableMemoryBuffer> Buf;
-
-public:
-  Error allocate(size_t Size) override;
-  uint8_t *getBufferStart() override;
-  Error commit() override;
-
-  explicit MemBuffer(StringRef Name) : Buffer(Name) {}
-
-  std::unique_ptr<WritableMemoryBuffer> releaseMemoryBuffer();
-};
-
-} // end namespace objcopy
-} // end namespace llvm
-
-#endif // LLVM_TOOLS_OBJCOPY_BUFFER_H

llvm/tools/llvm-objcopy/Buffer.cpp

-//===- Buffer.cpp ---------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-#include "Buffer.h"
-#include "llvm/Support/FileOutputBuffer.h"
-#include "llvm/Support/FileSystem.h"
-#include "llvm/Support/MemoryBuffer.h"
-#include "llvm/Support/Process.h"
-#include <memory>
-
-namespace llvm {
-namespace objcopy {
-
-Buffer::~Buffer() {}
-
-static Error createEmptyFile(StringRef FileName) {
-  // Create an empty tempfile and atomically swap it in place with the desired
-  // output file.
-  Expected<sys::fs::TempFile> Temp =
-      sys::fs::TempFile::create(FileName + ".temp-empty-%%%%%%%");
-  return Temp ? Temp->keep(FileName) : Temp.takeError();
-}
-
-Error FileBuffer::allocate(size_t Size) {
-  // When a 0-sized file is requested, skip allocation but defer file
-  // creation/truncation until commit() to avoid side effects if something
-  // happens between allocate() and commit().
-  if (Size == 0) {
-    EmptyFile = true;
-    return Error::success();
-  }
-
-  Expected<std::unique_ptr<FileOutputBuffer>> BufferOrErr =
-      FileOutputBuffer::create(getName(), Size, FileOutputBuffer::F_executable);
-  // FileOutputBuffer::create() returns an Error that is just a wrapper around
-  // std::error_code. Wrap it in FileError to include the actual filename.
-  if (!BufferOrErr)
-    return createFileError(getName(), BufferOrErr.takeError());
-  Buf = std::move(*BufferOrErr);
-  return Error::success();
-}
-
-Error FileBuffer::commit() {
-  if (EmptyFile)
-    return createEmptyFile(getName());
-
-  assert(Buf && "allocate() not called before commit()!");
-  Error Err = Buf->commit();
-  // FileOutputBuffer::commit() returns an Error that is just a wrapper around
-  // std::error_code. Wrap it in FileError to include the actual filename.
-  return Err ? createFileError(getName(), std::move(Err)) : std::move(Err);
-}
-
-uint8_t *FileBuffer::getBufferStart() {
-  return reinterpret_cast<uint8_t *>(Buf->getBufferStart());
-}
-
-Error MemBuffer::allocate(size_t Size) {
-  Buf = WritableMemoryBuffer::getNewMemBuffer(Size, getName());
-  return Error::success();
-}
-
-Error MemBuffer::commit() { return Error::success(); }
-
-uint8_t *MemBuffer::getBufferStart() {
-  return reinterpret_cast<uint8_t *>(Buf->getBufferStart());
-}
-
-std::unique_ptr<WritableMemoryBuffer> MemBuffer::releaseMemoryBuffer() {
-  return std::move(Buf);
-}
-
-} // end namespace objcopy
-} // end namespace llvm

llvm/tools/llvm-objcopy/CMakeLists.txt

 tablegen(LLVM BitcodeStripOpts.inc -gen-opt-parser-defs)
 add_public_tablegen_target(BitcodeStripOptsTableGen)
 
 set(LLVM_TARGET_DEFINITIONS StripOpts.td)
 tablegen(LLVM StripOpts.inc -gen-opt-parser-defs)
 add_public_tablegen_target(StripOptsTableGen)
 
 add_llvm_tool(llvm-objcopy
-  Buffer.cpp
   CopyConfig.cpp
   llvm-objcopy.cpp
   COFF/COFFObjcopy.cpp
   COFF/Object.cpp
   COFF/Reader.cpp
   COFF/Writer.cpp
   ELF/ELFConfig.cpp
   ELF/ELFObjcopy.cpp

llvm/tools/llvm-objcopy/COFF/COFFObjcopy.h

 //===- COFFObjcopy.h --------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TOOLS_OBJCOPY_COFFOBJCOPY_H
 #define LLVM_TOOLS_OBJCOPY_COFFOBJCOPY_H
 
 namespace llvm {
 class Error;
+class raw_ostream;
 
 namespace object {
 class COFFObjectFile;
 } // end namespace object
 
 namespace objcopy {
 struct CopyConfig;
-class Buffer;
 
 namespace coff {
 Error executeObjcopyOnBinary(const CopyConfig &Config,
-                             object::COFFObjectFile &In, Buffer &Out);
+                             object::COFFObjectFile &In, raw_ostream &Out);
 
 } // end namespace coff
 } // end namespace objcopy
 } // end namespace llvm
 
 #endif // LLVM_TOOLS_OBJCOPY_COFFOBJCOPY_H

llvm/tools/llvm-objcopy/COFF/COFFObjcopy.cpp

 //===- COFFObjcopy.cpp ----------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "COFFObjcopy.h"
-#include "Buffer.h"
 #include "CopyConfig.h"
 #include "Object.h"
 #include "Reader.h"
 #include "Writer.h"
 
 #include "llvm/Object/Binary.h"
 #include "llvm/Object/COFF.h"
 #include "llvm/Support/CRC.h"
if (Config.AllowBrokenLinks || !Config.BuildIdLinkDir.empty() ||
     return createStringError(llvm::errc::invalid_argument,
                              "option not supported by llvm-objcopy for COFF");
   }
 
   return Error::success();
 }
 
 Error executeObjcopyOnBinary(const CopyConfig &Config, COFFObjectFile &In,
-                             Buffer &Out) {
+                             raw_ostream &Out) {
   COFFReader Reader(In);
   Expected<std::unique_ptr<Object>> ObjOrErr = Reader.create();
   if (!ObjOrErr)
     return createFileError(Config.InputFilename, ObjOrErr.takeError());
   Object *Obj = ObjOrErr->get();
   assert(Obj && "Unable to deserialize COFF object");
   if (Error E = handleArgs(Config, *Obj))
     return createFileError(Config.InputFilename, std::move(E));

llvm/tools/llvm-objcopy/COFF/Reader.h

 //===- Reader.h -------------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TOOLS_OBJCOPY_COFF_READER_H
 #define LLVM_TOOLS_OBJCOPY_COFF_READER_H
 
-#include "Buffer.h"
 #include "llvm/BinaryFormat/COFF.h"
 #include "llvm/Object/COFF.h"
 #include "llvm/Support/Error.h"
 
 namespace llvm {
 namespace objcopy {
 namespace coff {
 

llvm/tools/llvm-objcopy/COFF/Writer.h

 //===- Writer.h -------------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TOOLS_OBJCOPY_COFF_WRITER_H
 #define LLVM_TOOLS_OBJCOPY_COFF_WRITER_H
 
-#include "Buffer.h"
 #include "llvm/MC/StringTableBuilder.h"
 #include "llvm/Support/Error.h"
+#include "llvm/Support/MemoryBuffer.h"
 #include <cstddef>
 #include <utility>
 
 namespace llvm {
 namespace objcopy {
 namespace coff {
 
 struct Object;
 
 class COFFWriter {
   Object &Obj;
-  Buffer &Buf;
+  std::unique_ptr<WritableMemoryBuffer> Buf;
+  raw_ostream &Out;
 
   size_t FileSize;
   size_t FileAlignment;
   size_t SizeOfInitializedData;
   StringTableBuilder StrTabBuilder;
 
   template <class SymbolTy> std::pair<size_t, size_t> finalizeSymbolTable();
   Error finalizeRelocTargets();
class COFFWriter {
 
   Error patchDebugDirectory();
   Expected<uint32_t> virtualAddressToFileAddress(uint32_t RVA);
 
 public:
   virtual ~COFFWriter() {}
   Error write();
 
-  COFFWriter(Object &Obj, Buffer &Buf)
-      : Obj(Obj), Buf(Buf), StrTabBuilder(StringTableBuilder::WinCOFF) {}
+  COFFWriter(Object &Obj, raw_ostream &Out)
+      : Obj(Obj), Out(Out), StrTabBuilder(StringTableBuilder::WinCOFF) {}
 };
 
 } // end namespace coff
 } // end namespace objcopy
 } // end namespace llvm
 
 #endif // LLVM_TOOLS_OBJCOPY_COFF_WRITER_H

llvm/tools/llvm-objcopy/COFF/Writer.cpp

 //===- Writer.cpp ---------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "Writer.h"
 #include "Object.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/BinaryFormat/COFF.h"
 #include "llvm/Object/COFF.h"
+#include "llvm/Support/Errc.h"
 #include "llvm/Support/ErrorHandling.h"
 #include <cstddef>
 #include <cstdint>
 
 namespace llvm {
 namespace objcopy {
 namespace coff {
 
Error COFFWriter::finalize(bool IsBigObj) {
   Obj.CoffFileHeader.NumberOfSymbols = NumRawSymbols;
   FileSize += SymTabSize + StrTabSize;
   FileSize = alignTo(FileSize, FileAlignment);
 
   return Error::success();
 }
 
 void COFFWriter::writeHeaders(bool IsBigObj) {
-  uint8_t *Ptr = Buf.getBufferStart();
+  uint8_t *Ptr = reinterpret_cast<uint8_t *>(Buf->getBufferStart());
   if (Obj.IsPE) {
     memcpy(Ptr, &Obj.DosHeader, sizeof(Obj.DosHeader));
     Ptr += sizeof(Obj.DosHeader);
     memcpy(Ptr, Obj.DosStub.data(), Obj.DosStub.size());
     Ptr += Obj.DosStub.size();
     memcpy(Ptr, PEMagic, sizeof(PEMagic));
     Ptr += sizeof(PEMagic);
   }
void COFFWriter::writeHeaders(bool IsBigObj) {
   for (const auto &S : Obj.getSections()) {
     memcpy(Ptr, &S.Header, sizeof(S.Header));
     Ptr += sizeof(S.Header);
   }
 }
 
 void COFFWriter::writeSections() {
   for (const auto &S : Obj.getSections()) {
-    uint8_t *Ptr = Buf.getBufferStart() + S.Header.PointerToRawData;
+    uint8_t *Ptr = reinterpret_cast<uint8_t *>(Buf->getBufferStart()) +
+                   S.Header.PointerToRawData;
     ArrayRef<uint8_t> Contents = S.getContents();
     std::copy(Contents.begin(), Contents.end(), Ptr);
 
     // For executable sections, pad the remainder of the raw data size with
     // 0xcc, which is int3 on x86.
     if ((S.Header.Characteristics & IMAGE_SCN_CNT_CODE) &&
         S.Header.SizeOfRawData > Contents.size())
       memset(Ptr + Contents.size(), 0xcc,
for (const auto &S : Obj.getSections()) {
     for (const auto &R : S.Relocs) {
       memcpy(Ptr, &R.Reloc, sizeof(R.Reloc));
       Ptr += sizeof(R.Reloc);
     }
   }
 }
 
 template <class SymbolTy> void COFFWriter::writeSymbolStringTables() {
-  uint8_t *Ptr = Buf.getBufferStart() + Obj.CoffFileHeader.PointerToSymbolTable;
+  uint8_t *Ptr = reinterpret_cast<uint8_t *>(Buf->getBufferStart()) +
+                 Obj.CoffFileHeader.PointerToSymbolTable;
   for (const auto &S : Obj.getSymbols()) {
     // Convert symbols back to the right size, from coff_symbol32.
     copySymbol<SymbolTy, coff_symbol32>(*reinterpret_cast<SymbolTy *>(Ptr),
                                         S.Sym);
     Ptr += sizeof(SymbolTy);
     if (!S.AuxFile.empty()) {
       // For file symbols, just write the string into the aux symbol slots,
       // assuming that the unwritten parts are initialized to zero in the memory
if (StrTabBuilder.getSize() > 4 || !Obj.IsPE) {
     Ptr += StrTabBuilder.getSize();
   }
 }
 
 Error COFFWriter::write(bool IsBigObj) {
   if (Error E = finalize(IsBigObj))
     return E;
 
-  if (Error E = Buf.allocate(FileSize))
-    return E;
+  Buf = WritableMemoryBuffer::getNewMemBuffer(FileSize);
+  if (!Buf)
+    return createStringError(llvm::errc::not_enough_memory, Twine(FileSize));
 
   writeHeaders(IsBigObj);
   writeSections();
   if (IsBigObj)
     writeSymbolStringTables<coff_symbol32>();
   else
     writeSymbolStringTables<coff_symbol16>();
 
   if (Obj.IsPE)
     if (Error E = patchDebugDirectory())
       return E;
 
-  return Buf.commit();
+  // TODO: Implement direct writing to the output stream
+  // TODO: (without intermediate memory buffer Buf).
+  Out.write(Buf->getBufferStart(), Buf->getBufferSize());
+  Out.flush();
+  return Error::success();
 }
 
 Expected<uint32_t> COFFWriter::virtualAddressToFileAddress(uint32_t RVA) {
   for (const auto &S : Obj.getSections()) {
     if (RVA >= S.Header.VirtualAddress &&
         RVA < S.Header.VirtualAddress + S.Header.SizeOfRawData)
       return S.Header.PointerToRawData + RVA - S.Header.VirtualAddress;
   }
if (Dir->RelativeVirtualAddress >= S.Header.VirtualAddress &&
         Dir->RelativeVirtualAddress <
             S.Header.VirtualAddress + S.Header.SizeOfRawData) {
       if (Dir->RelativeVirtualAddress + Dir->Size >
           S.Header.VirtualAddress + S.Header.SizeOfRawData)
         return createStringError(object_error::parse_failed,
                                  "debug directory extends past end of section");
 
       size_t Offset = Dir->RelativeVirtualAddress - S.Header.VirtualAddress;
-      uint8_t *Ptr = Buf.getBufferStart() + S.Header.PointerToRawData + Offset;
+      uint8_t *Ptr = reinterpret_cast<uint8_t *>(Buf->getBufferStart()) +
+                     S.Header.PointerToRawData + Offset;
       uint8_t *End = Ptr + Dir->Size;
       while (Ptr < End) {
         debug_directory *Debug = reinterpret_cast<debug_directory *>(Ptr);
         if (!Debug->AddressOfRawData)
           return createStringError(object_error::parse_failed,
                                    "debug directory payload outside of "
                                    "mapped sections not supported");
         if (Expected<uint32_t> FilePosOrErr =

llvm/tools/llvm-objcopy/ELF/ELFObjcopy.h

 //===- ELFObjcopy.h ---------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TOOLS_OBJCOPY_ELFOBJCOPY_H
 #define LLVM_TOOLS_OBJCOPY_ELFOBJCOPY_H
 
 namespace llvm {
 class Error;
 class MemoryBuffer;
+class raw_ostream;
 
 namespace object {
 class ELFObjectFileBase;
 } // end namespace object
 
 namespace objcopy {
 struct CopyConfig;
-class Buffer;
 
 namespace elf {
 Error executeObjcopyOnIHex(const CopyConfig &Config, MemoryBuffer &In,
-                           Buffer &Out);
+                           raw_ostream &Out);
 Error executeObjcopyOnRawBinary(const CopyConfig &Config, MemoryBuffer &In,
-                                Buffer &Out);
+                                raw_ostream &Out);
 Error executeObjcopyOnBinary(const CopyConfig &Config,
-                             object::ELFObjectFileBase &In, Buffer &Out);
+                             object::ELFObjectFileBase &In, raw_ostream &Out);
 
 } // end namespace elf
 } // end namespace objcopy
 } // end namespace llvm
 
 #endif // LLVM_TOOLS_OBJCOPY_ELFOBJCOPY_H

llvm/tools/llvm-objcopy/ELF/ELFObjcopy.cpp

 //===- ELFObjcopy.cpp -----------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "ELFObjcopy.h"
-#include "Buffer.h"
 #include "CopyConfig.h"
 #include "Object.h"
+#include "Util.h"
 #include "llvm/ADT/BitmaskEnum.h"
 #include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/BinaryFormat/ELF.h"
static ElfType getOutputElfType(const MachineInfo &MI) {
   // Infer output ELF type from the binary arch specified
   if (MI.Is64Bit)
     return MI.IsLittleEndian ? ELFT_ELF64LE : ELFT_ELF64BE;
   else
     return MI.IsLittleEndian ? ELFT_ELF32LE : ELFT_ELF32BE;
 }
 
 static std::unique_ptr<Writer> createELFWriter(const CopyConfig &Config,
-                                               Object &Obj, Buffer &Buf,
+                                               Object &Obj, raw_ostream &Out,
                                                ElfType OutputElfType) {
   // Depending on the initial ELFT and OutputFormat we need a different Writer.
   switch (OutputElfType) {
   case ELFT_ELF32LE:
-    return std::make_unique<ELFWriter<ELF32LE>>(Obj, Buf, !Config.StripSections,
+    return std::make_unique<ELFWriter<ELF32LE>>(Obj, Out, !Config.StripSections,
                                                 Config.OnlyKeepDebug);
   case ELFT_ELF64LE:
-    return std::make_unique<ELFWriter<ELF64LE>>(Obj, Buf, !Config.StripSections,
+    return std::make_unique<ELFWriter<ELF64LE>>(Obj, Out, !Config.StripSections,
                                                 Config.OnlyKeepDebug);
   case ELFT_ELF32BE:
-    return std::make_unique<ELFWriter<ELF32BE>>(Obj, Buf, !Config.StripSections,
+    return std::make_unique<ELFWriter<ELF32BE>>(Obj, Out, !Config.StripSections,
                                                 Config.OnlyKeepDebug);
   case ELFT_ELF64BE:
-    return std::make_unique<ELFWriter<ELF64BE>>(Obj, Buf, !Config.StripSections,
+    return std::make_unique<ELFWriter<ELF64BE>>(Obj, Out, !Config.StripSections,
                                                 Config.OnlyKeepDebug);
   }
   llvm_unreachable("Invalid output format");
 }
 
 static std::unique_ptr<Writer> createWriter(const CopyConfig &Config,
-                                            Object &Obj, Buffer &Buf,
+                                            Object &Obj, raw_ostream &Out,
                                             ElfType OutputElfType) {
   switch (Config.OutputFormat) {
   case FileFormat::Binary:
-    return std::make_unique<BinaryWriter>(Obj, Buf);
+    return std::make_unique<BinaryWriter>(Obj, Out);
   case FileFormat::IHex:
-    return std::make_unique<IHexWriter>(Obj, Buf);
+    return std::make_unique<IHexWriter>(Obj, Out);
   default:
-    return createELFWriter(Config, Obj, Buf, OutputElfType);
+    return createELFWriter(Config, Obj, Out, OutputElfType);
   }
 }
 
-template <class ELFT>
-static Expected<ArrayRef<uint8_t>>
-findBuildID(const CopyConfig &Config, const object::ELFFile<ELFT> &In) {
-  auto PhdrsOrErr = In.program_headers();
-  if (auto Err = PhdrsOrErr.takeError())
-    return createFileError(Config.InputFilename, std::move(Err));
-
-  for (const auto &Phdr : *PhdrsOrErr) {
-    if (Phdr.p_type != PT_NOTE)
-      continue;
-    Error Err = Error::success();
-    for (auto Note : In.notes(Phdr, Err))
-      if (Note.getType() == NT_GNU_BUILD_ID && Note.getName() == ELF_NOTE_GNU)
-        return Note.getDesc();
-    if (Err)
-      return createFileError(Config.InputFilename, std::move(Err));
-  }
-
-  return createFileError(Config.InputFilename,
-                         createStringError(llvm::errc::invalid_argument,
-                                           "could not find build ID"));
-}
-
-static Expected<ArrayRef<uint8_t>>
-findBuildID(const CopyConfig &Config, const object::ELFObjectFileBase &In) {
-  if (auto *O = dyn_cast<ELFObjectFile<ELF32LE>>(&In))
-    return findBuildID(Config, O->getELFFile());
-  else if (auto *O = dyn_cast<ELFObjectFile<ELF64LE>>(&In))
-    return findBuildID(Config, O->getELFFile());
-  else if (auto *O = dyn_cast<ELFObjectFile<ELF32BE>>(&In))
-    return findBuildID(Config, O->getELFFile());
-  else if (auto *O = dyn_cast<ELFObjectFile<ELF64BE>>(&In))
-    return findBuildID(Config, O->getELFFile());
-
-  llvm_unreachable("Bad file format");
-}
-
-template <class... Ts>
-static Error makeStringError(std::error_code EC, const Twine &Msg,
-                             Ts &&... Args) {
-  std::string FullMsg = (EC.message() + ": " + Msg).str();
-  return createStringError(EC, FullMsg.c_str(), std::forward<Ts>(Args)...);
-}
-
-#define MODEL_8 "%%%%%%%%"
-#define MODEL_16 MODEL_8 MODEL_8
-#define MODEL_32 (MODEL_16 MODEL_16)
-
-static Error linkToBuildIdDir(const CopyConfig &Config, StringRef ToLink,
-                              StringRef Suffix,
-                              ArrayRef<uint8_t> BuildIdBytes) {
-  SmallString<128> Path = Config.BuildIdLinkDir;
-  sys::path::append(Path, llvm::toHex(BuildIdBytes[0], /*LowerCase*/ true));
-  if (auto EC = sys::fs::create_directories(Path))
-    return createFileError(
-        Path.str(),
-        makeStringError(EC, "cannot create build ID link directory"));
-
-  sys::path::append(Path,
-                    llvm::toHex(BuildIdBytes.slice(1), /*LowerCase*/ true));
-  Path += Suffix;
-  SmallString<128> TmpPath;
-  // create_hard_link races so we need to link to a temporary path but
-  // we want to make sure that we choose a filename that does not exist.
-  // By using 32 model characters we get 128-bits of entropy. It is
-  // unlikely that this string has ever existed before much less exists
-  // on this disk or in the current working directory.
-  // Additionally we prepend the original Path for debugging but also
-  // because it ensures that we're linking within a directory on the same
-  // partition on the same device which is critical. It has the added
-  // win of yet further decreasing the odds of a conflict.
-  sys::fs::createUniquePath(Twine(Path) + "-" + MODEL_32 + ".tmp", TmpPath,
-                            /*MakeAbsolute*/ false);
-  if (auto EC = sys::fs::create_hard_link(ToLink, TmpPath)) {
-    Path.push_back('\0');
-    return makeStringError(EC, "cannot link '%s' to '%s'", ToLink.data(),
-                           Path.data());
-  }
-  // We then atomically rename the link into place which will just move the
-  // link. If rename fails something is more seriously wrong so just return
-  // an error.
-  if (auto EC = sys::fs::rename(TmpPath, Path)) {
-    Path.push_back('\0');
-    return makeStringError(EC, "cannot link '%s' to '%s'", ToLink.data(),
-                           Path.data());
-  }
-  // If `Path` was already a hard-link to the same underlying file then the
-  // temp file will be left so we need to remove it. Remove will not cause
-  // an error by default if the file is already gone so just blindly remove
-  // it rather than checking.
-  if (auto EC = sys::fs::remove(TmpPath)) {
-    TmpPath.push_back('\0');
-    return makeStringError(EC, "could not remove '%s'", TmpPath.data());
-  }
-  return Error::success();
-}
-
 static Error splitDWOToFile(const CopyConfig &Config, const Reader &Reader,
                             StringRef File, ElfType OutputElfType) {
   Expected<std::unique_ptr<Object>> DWOFile = Reader.create(false);
   if (!DWOFile)
     return DWOFile.takeError();
 
   auto OnlyKeepDWOPred = [&DWOFile](const SectionBase &Sec) {
     return onlyKeepDWOPred(**DWOFile, Sec);
   };
   if (Error E =
           (*DWOFile)->removeSections(Config.AllowBrokenLinks, OnlyKeepDWOPred))
     return E;
   if (Config.OutputArch) {
     (*DWOFile)->Machine = Config.OutputArch.getValue().EMachine;
     (*DWOFile)->OSABI = Config.OutputArch.getValue().OSABI;
   }
-  FileBuffer FB(File);
+
+  return writeToFile(File, [&](raw_ostream &OutFile) -> Error {
     std::unique_ptr<Writer> Writer =
-      createWriter(Config, **DWOFile, FB, OutputElfType);
+        createWriter(Config, **DWOFile, OutFile, OutputElfType);
     if (Error E = Writer->finalize())
       return E;
     return Writer->write();
+  });
 }
 
 static Error dumpSectionToFile(StringRef SecName, StringRef Filename,
                                Object &Obj) {
   for (auto &Sec : Obj.sections()) {
     if (Sec.Name == SecName) {
       if (Sec.Type == SHT_NOBITS)
         return createStringError(object_error::parse_failed,
if (!Config.SetSectionFlags.empty()) {
     }
   }
 
   if (Config.EntryExpr)
     Obj.Entry = Config.EntryExpr(Obj.Entry);
   return Error::success();
 }
 
-static Error writeOutput(const CopyConfig &Config, Object &Obj, Buffer &Out,
-                         ElfType OutputElfType) {
+static Error writeOutput(const CopyConfig &Config, Object &Obj,
+                         raw_ostream &Out, ElfType OutputElfType) {
   std::unique_ptr<Writer> Writer =
       createWriter(Config, Obj, Out, OutputElfType);
   if (Error E = Writer->finalize())
     return E;
   return Writer->write();
 }
 
 Error executeObjcopyOnIHex(const CopyConfig &Config, MemoryBuffer &In,
-                           Buffer &Out) {
+                           raw_ostream &Out) {
   IHexReader Reader(&In);
   Expected<std::unique_ptr<Object>> Obj = Reader.create(true);
   if (!Obj)
     return Obj.takeError();
 
   const ElfType OutputElfType =
       getOutputElfType(Config.OutputArch.getValueOr(MachineInfo()));
   if (Error E = handleArgs(Config, **Obj, Reader, OutputElfType))
     return E;
   return writeOutput(Config, **Obj, Out, OutputElfType);
 }
 
 Error executeObjcopyOnRawBinary(const CopyConfig &Config, MemoryBuffer &In,
-                                Buffer &Out) {
+                                raw_ostream &Out) {
   uint8_t NewSymbolVisibility =
       Config.ELF->NewSymbolVisibility.getValueOr((uint8_t)ELF::STV_DEFAULT);
   BinaryReader Reader(&In, NewSymbolVisibility);
   Expected<std::unique_ptr<Object>> Obj = Reader.create(true);
   if (!Obj)
     return Obj.takeError();
 
   // Prefer OutputArch (-O<format>) if set, otherwise fallback to BinaryArch
   // (-B<arch>).
   const ElfType OutputElfType =
       getOutputElfType(Config.OutputArch.getValueOr(MachineInfo()));
   if (Error E = handleArgs(Config, **Obj, Reader, OutputElfType))
     return E;
   return writeOutput(Config, **Obj, Out, OutputElfType);
 }
 
 Error executeObjcopyOnBinary(const CopyConfig &Config,
-                             object::ELFObjectFileBase &In, Buffer &Out) {
+                             object::ELFObjectFileBase &In, raw_ostream &Out) {
   ELFReader Reader(&In, Config.ExtractPartition);
   Expected<std::unique_ptr<Object>> Obj =
       Reader.create(!Config.SymbolsToAdd.empty());
   if (!Obj)
     return Obj.takeError();
   // Prefer OutputArch (-O<format>) if set, otherwise infer it from the input.
   const ElfType OutputElfType =
       Config.OutputArch ? getOutputElfType(Config.OutputArch.getValue())
                         : getOutputElfType(In);
-  ArrayRef<uint8_t> BuildIdBytes;
-
-  if (!Config.BuildIdLinkDir.empty()) {
-    auto BuildIdBytesOrErr = findBuildID(Config, In);
-    if (auto E = BuildIdBytesOrErr.takeError())
-      return E;
-    BuildIdBytes = *BuildIdBytesOrErr;
-
-    if (BuildIdBytes.size() < 2)
-      return createFileError(
-          Config.InputFilename,
-          createStringError(object_error::parse_failed,
-                            "build ID is smaller than two bytes"));
-  }
-
-  if (!Config.BuildIdLinkDir.empty() && Config.BuildIdLinkInput)
-    if (Error E =
-            linkToBuildIdDir(Config, Config.InputFilename,
-                             Config.BuildIdLinkInput.getValue(), BuildIdBytes))
-      return E;
 
   if (Error E = handleArgs(Config, **Obj, Reader, OutputElfType))
     return createFileError(Config.InputFilename, std::move(E));
 
   if (Error E = writeOutput(Config, **Obj, Out, OutputElfType))
     return createFileError(Config.InputFilename, std::move(E));
-  if (!Config.BuildIdLinkDir.empty() && Config.BuildIdLinkOutput)
-    if (Error E =
-            linkToBuildIdDir(Config, Config.OutputFilename,
-                             Config.BuildIdLinkOutput.getValue(), BuildIdBytes))
-      return createFileError(Config.OutputFilename, std::move(E));
 
   return Error::success();
 }
 
 } // end namespace elf
 } // end namespace objcopy
 } // end namespace llvm

llvm/tools/llvm-objcopy/ELF/Object.h

 //===- Object.h -------------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TOOLS_OBJCOPY_OBJECT_H
 #define LLVM_TOOLS_OBJCOPY_OBJECT_H
 
-#include "Buffer.h"
 #include "CopyConfig.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/BinaryFormat/ELF.h"
 #include "llvm/MC/StringTableBuilder.h"
 #include "llvm/Object/ELFObjectFile.h"
 #include "llvm/Support/Errc.h"
 #include "llvm/Support/FileOutputBuffer.h"
+#include "llvm/Support/MemoryBuffer.h"
 #include <cstddef>
 #include <cstdint>
 #include <functional>
 #include <memory>
 #include <set>
 #include <vector>
 
 namespace llvm {
public:
   virtual Error visit(GroupSection &Sec) = 0;
   virtual Error visit(SectionIndexSection &Sec) = 0;
   virtual Error visit(CompressedSection &Sec) = 0;
   virtual Error visit(DecompressedSection &Sec) = 0;
 };
 
 class SectionWriter : public SectionVisitor {
 protected:
-  Buffer &Out;
+  WritableMemoryBuffer &Out;
 
 public:
   virtual ~SectionWriter() = default;
 
   Error visit(const Section &Sec) override;
   Error visit(const OwnedDataSection &Sec) override;
   Error visit(const StringTableSection &Sec) override;
   Error visit(const DynamicRelocationSection &Sec) override;
   virtual Error visit(const SymbolTableSection &Sec) override = 0;
   virtual Error visit(const RelocationSection &Sec) override = 0;
   virtual Error visit(const GnuDebugLinkSection &Sec) override = 0;
   virtual Error visit(const GroupSection &Sec) override = 0;
   virtual Error visit(const SectionIndexSection &Sec) override = 0;
   virtual Error visit(const CompressedSection &Sec) override = 0;
   virtual Error visit(const DecompressedSection &Sec) override = 0;
 
-  explicit SectionWriter(Buffer &Buf) : Out(Buf) {}
+  explicit SectionWriter(WritableMemoryBuffer &Buf) : Out(Buf) {}
 };
 
 template <class ELFT> class ELFSectionWriter : public SectionWriter {
 private:
   using Elf_Word = typename ELFT::Word;
   using Elf_Rel = typename ELFT::Rel;
   using Elf_Rela = typename ELFT::Rela;
   using Elf_Sym = typename ELFT::Sym;
 
 public:
   virtual ~ELFSectionWriter() {}
   Error visit(const SymbolTableSection &Sec) override;
   Error visit(const RelocationSection &Sec) override;
   Error visit(const GnuDebugLinkSection &Sec) override;
   Error visit(const GroupSection &Sec) override;
   Error visit(const SectionIndexSection &Sec) override;
   Error visit(const CompressedSection &Sec) override;
   Error visit(const DecompressedSection &Sec) override;
 
-  explicit ELFSectionWriter(Buffer &Buf) : SectionWriter(Buf) {}
+  explicit ELFSectionWriter(WritableMemoryBuffer &Buf) : SectionWriter(Buf) {}
 };
 
 template <class ELFT> class ELFSectionSizer : public MutableSectionVisitor {
 private:
   using Elf_Rel = typename ELFT::Rel;
   using Elf_Rela = typename ELFT::Rela;
   using Elf_Sym = typename ELFT::Sym;
   using Elf_Word = typename ELFT::Word;
public:
   Error visit(const SymbolTableSection &Sec) override;
   Error visit(const RelocationSection &Sec) override;
   Error visit(const GnuDebugLinkSection &Sec) override;
   Error visit(const GroupSection &Sec) override;
   Error visit(const SectionIndexSection &Sec) override;
   Error visit(const CompressedSection &Sec) override;
   Error visit(const DecompressedSection &Sec) override;
 
-  explicit BinarySectionWriter(Buffer &Buf) : SectionWriter(Buf) {}
+  explicit BinarySectionWriter(WritableMemoryBuffer &Buf)
+      : SectionWriter(Buf) {}
 };
 
 using IHexLineData = SmallVector<char, 64>;
 
 struct IHexRecord {
   // Memory address of the record.
   uint16_t Addr;
   // Record type (see below).

 protected:
   // Offset in the output buffer
   uint64_t Offset = 0;
 
   void writeSection(const SectionBase *Sec, ArrayRef<uint8_t> Data);
   virtual void writeData(uint8_t Type, uint16_t Addr, ArrayRef<uint8_t> Data);
 
 public:
-  explicit IHexSectionWriterBase(Buffer &Buf) : BinarySectionWriter(Buf) {}
+  explicit IHexSectionWriterBase(WritableMemoryBuffer &Buf)
+      : BinarySectionWriter(Buf) {}
 
   uint64_t getBufferOffset() const { return Offset; }
   Error visit(const Section &Sec) final;
   Error visit(const OwnedDataSection &Sec) final;
   Error visit(const StringTableSection &Sec) override;
   Error visit(const DynamicRelocationSection &Sec) final;
   using BinarySectionWriter::visit;
 };
 
 // Real IHEX section writer
 class IHexSectionWriter : public IHexSectionWriterBase {
 public:
-  IHexSectionWriter(Buffer &Buf) : IHexSectionWriterBase(Buf) {}
+  IHexSectionWriter(WritableMemoryBuffer &Buf) : IHexSectionWriterBase(Buf) {}
 
   void writeData(uint8_t Type, uint16_t Addr, ArrayRef<uint8_t> Data) override;
   Error visit(const StringTableSection &Sec) override;
 };
 
 class Writer {
 protected:
   Object &Obj;
-  Buffer &Buf;
+  std::unique_ptr<WritableMemoryBuffer> Buf;
+  raw_ostream &Out;
 
 public:
   virtual ~Writer();
   virtual Error finalize() = 0;
   virtual Error write() = 0;
 
-  Writer(Object &O, Buffer &B) : Obj(O), Buf(B) {}
+  Writer(Object &O, raw_ostream &Out) : Obj(O), Out(Out) {}
 };
 
 template <class ELFT> class ELFWriter : public Writer {
 private:
   using Elf_Addr = typename ELFT::Addr;
   using Elf_Shdr = typename ELFT::Shdr;
   using Elf_Phdr = typename ELFT::Phdr;
   using Elf_Ehdr = typename ELFT::Ehdr;
public:
   bool WriteSectionHeaders;
 
   // For --only-keep-debug, select an alternative section/segment layout
   // algorithm.
   bool OnlyKeepDebug;
 
   Error finalize() override;
   Error write() override;
-  ELFWriter(Object &Obj, Buffer &Buf, bool WSH, bool OnlyKeepDebug);
+  ELFWriter(Object &Obj, raw_ostream &Out, bool WSH, bool OnlyKeepDebug);
 };
 
 class BinaryWriter : public Writer {
 private:
   std::unique_ptr<BinarySectionWriter> SecWriter;
 
   uint64_t TotalSize = 0;
 
 public:
   ~BinaryWriter() {}
   Error finalize() override;
   Error write() override;
-  BinaryWriter(Object &Obj, Buffer &Buf) : Writer(Obj, Buf) {}
+  BinaryWriter(Object &Obj, raw_ostream &Out) : Writer(Obj, Out) {}
 };
 
 class IHexWriter : public Writer {
   struct SectionCompare {
     bool operator()(const SectionBase *Lhs, const SectionBase *Rhs) const;
   };
 
   std::set<const SectionBase *, SectionCompare> Sections;
   size_t TotalSize = 0;
 
   Error checkSection(const SectionBase &Sec);
   uint64_t writeEntryPointRecord(uint8_t *Buf);
   uint64_t writeEndOfFileRecord(uint8_t *Buf);
 
 public:
   ~IHexWriter() {}
   Error finalize() override;
   Error write() override;
-  IHexWriter(Object &Obj, Buffer &Buf) : Writer(Obj, Buf) {}
+  IHexWriter(Object &Obj, raw_ostream &Out) : Writer(Obj, Out) {}
 };
 
 class SectionBase {
 public:
   std::string Name;
   Segment *ParentSegment = nullptr;
   uint64_t HeaderOffset = 0;
   uint32_t Index = 0;

llvm/tools/llvm-objcopy/ELF/Object.cpp

 namespace llvm {
 namespace objcopy {
 namespace elf {
 
 using namespace object;
 using namespace ELF;
 
 template <class ELFT> void ELFWriter<ELFT>::writePhdr(const Segment &Seg) {
-  uint8_t *B = Buf.getBufferStart() + Obj.ProgramHdrSegment.Offset +
-               Seg.Index * sizeof(Elf_Phdr);
+  uint8_t *B = reinterpret_cast<uint8_t *>(Buf->getBufferStart()) +
+               Obj.ProgramHdrSegment.Offset + Seg.Index * sizeof(Elf_Phdr);
   Elf_Phdr &Phdr = *reinterpret_cast<Elf_Phdr *>(B);
   Phdr.p_type = Seg.Type;
   Phdr.p_flags = Seg.Flags;
   Phdr.p_offset = Seg.Offset;
   Phdr.p_vaddr = Seg.VAddr;
   Phdr.p_paddr = Seg.PAddr;
   Phdr.p_filesz = Seg.FileSize;
   Phdr.p_memsz = Seg.MemSize;

 Error SectionBase::initialize(SectionTableRef) { return Error::success(); }
 void SectionBase::finalize() {}
 void SectionBase::markSymbols() {}
 void SectionBase::replaceSectionReferences(
     const DenseMap<SectionBase *, SectionBase *> &) {}
 void SectionBase::onRemove() {}
 
 template <class ELFT> void ELFWriter<ELFT>::writeShdr(const SectionBase &Sec) {
-  uint8_t *B = Buf.getBufferStart() + Sec.HeaderOffset;
+  uint8_t *B =
+      reinterpret_cast<uint8_t *>(Buf->getBufferStart()) + Sec.HeaderOffset;
   Elf_Shdr &Shdr = *reinterpret_cast<Elf_Shdr *>(B);
   Shdr.sh_name = Sec.NameIndex;
   Shdr.sh_type = Sec.Type;
   Shdr.sh_flags = Sec.Flags;
   Shdr.sh_addr = Sec.Addr;
   Shdr.sh_offset = Sec.Offset;
   Shdr.sh_size = Sec.Size;
   Shdr.sh_link = Sec.Link;
StringRef CompressedContent(
       Sec.OriginalData.size() - DataOffset);
 
   SmallVector<char, 128> DecompressedContent;
   if (Error Err = zlib::uncompress(CompressedContent, DecompressedContent,
                                    static_cast<size_t>(Sec.Size)))
     return createStringError(errc::invalid_argument,
                              "'" + Sec.Name + "': " + toString(std::move(Err)));
 
-  uint8_t *Buf = Out.getBufferStart() + Sec.Offset;
+  uint8_t *Buf = reinterpret_cast<uint8_t *>(Out.getBufferStart()) + Sec.Offset;
   std::copy(DecompressedContent.begin(), DecompressedContent.end(), Buf);
 
   return Error::success();
 }
 
 Error BinarySectionWriter::visit(const DecompressedSection &Sec) {
   return createStringError(errc::operation_not_permitted,
                            "cannot write compressed section '" + Sec.Name +

 Error BinarySectionWriter::visit(const CompressedSection &Sec) {
   return createStringError(errc::operation_not_permitted,
                            "cannot write compressed section '" + Sec.Name +
                                "' ");
 }
 
 template <class ELFT>
 Error ELFSectionWriter<ELFT>::visit(const CompressedSection &Sec) {
-  uint8_t *Buf = Out.getBufferStart() + Sec.Offset;
+  uint8_t *Buf = reinterpret_cast<uint8_t *>(Out.getBufferStart()) + Sec.Offset;
   if (Sec.CompressionType == DebugCompressionType::None) {
     std::copy(Sec.OriginalData.begin(), Sec.OriginalData.end(), Buf);
     return Error::success();
   }
 
   if (Sec.CompressionType == DebugCompressionType::GNU) {
     const char *Magic = "ZLIB";
     memcpy(Buf, Magic, strlen(Magic));

 }
 
 void StringTableSection::prepareForLayout() {
   StrTabBuilder.finalize();
   Size = StrTabBuilder.getSize();
 }
 
 Error SectionWriter::visit(const StringTableSection &Sec) {
-  Sec.StrTabBuilder.write(Out.getBufferStart() + Sec.Offset);
+  Sec.StrTabBuilder.write(reinterpret_cast<uint8_t *>(Out.getBufferStart()) +
+                          Sec.Offset);
   return Error::success();
 }
 
 Error StringTableSection::accept(SectionVisitor &Visitor) const {
   return Visitor.visit(*this);
 }
 
 Error StringTableSection::accept(MutableSectionVisitor &Visitor) {
   return Visitor.visit(*this);
 }
 
 template <class ELFT>
 Error ELFSectionWriter<ELFT>::visit(const SectionIndexSection &Sec) {
-  uint8_t *Buf = Out.getBufferStart() + Sec.Offset;
+  uint8_t *Buf = reinterpret_cast<uint8_t *>(Out.getBufferStart()) + Sec.Offset;
   llvm::copy(Sec.Indexes, reinterpret_cast<Elf_Word *>(Buf));
   return Error::success();
 }
 
 Error SectionIndexSection::initialize(SectionTableRef SecTable) {
   Size = 0;
   Expected<SymbolTableSection *> Sec =
       SecTable.getSectionOfType<SymbolTableSection>(
for (const auto &Reloc : Relocations) {
     Buf->setSymbolAndType(Reloc.RelocSymbol ? Reloc.RelocSymbol->Index : 0,
                           Reloc.Type, false);
     ++Buf;
   }
 }
 
 template <class ELFT>
 Error ELFSectionWriter<ELFT>::visit(const RelocationSection &Sec) {
-  uint8_t *Buf = Out.getBufferStart() + Sec.Offset;
+  uint8_t *Buf = reinterpret_cast<uint8_t *>(Out.getBufferStart()) + Sec.Offset;
   if (Sec.Type == SHT_REL)
     writeRel(Sec.Relocations, reinterpret_cast<Elf_Rel *>(Buf));
   else
     writeRel(Sec.Relocations, reinterpret_cast<Elf_Rela *>(Buf));
   return Error::success();
 }
 
 Error RelocationSection::accept(SectionVisitor &Visitor) const {

 GnuDebugLinkSection::GnuDebugLinkSection(StringRef File,
                                          uint32_t PrecomputedCRC)
     : FileName(File), CRC32(PrecomputedCRC) {
   init(File);
 }
 
 template <class ELFT>
 Error ELFSectionWriter<ELFT>::visit(const GnuDebugLinkSection &Sec) {
-  unsigned char *Buf = Out.getBufferStart() + Sec.Offset;
+  unsigned char *Buf =
+      reinterpret_cast<uint8_t *>(Out.getBufferStart()) + Sec.Offset;
   Elf_Word *CRC =
       reinterpret_cast<Elf_Word *>(Buf + Sec.Size - sizeof(Elf_Word));
   *CRC = Sec.CRC32;
   llvm::copy(Sec.FileName, Buf);
   return Error::success();
 }
 
 Error GnuDebugLinkSection::accept(SectionVisitor &Visitor) const {
if (auto *O = dyn_cast<ELFObjectFile<ELF32LE>>(Bin)) {
     if (Error Err = Builder.build(EnsureSymtab))
       return std::move(Err);
     return std::move(Obj);
   }
   return createStringError(errc::invalid_argument, "invalid file type");
 }
 
 template <class ELFT> void ELFWriter<ELFT>::writeEhdr() {
-  Elf_Ehdr &Ehdr = *reinterpret_cast<Elf_Ehdr *>(Buf.getBufferStart());
+  Elf_Ehdr &Ehdr = *reinterpret_cast<Elf_Ehdr *>(Buf->getBufferStart());
   std::fill(Ehdr.e_ident, Ehdr.e_ident + 16, 0);
   Ehdr.e_ident[EI_MAG0] = 0x7f;
   Ehdr.e_ident[EI_MAG1] = 'E';
   Ehdr.e_ident[EI_MAG2] = 'L';
   Ehdr.e_ident[EI_MAG3] = 'F';
   Ehdr.e_ident[EI_CLASS] = ELFT::Is64Bits ? ELFCLASS64 : ELFCLASS32;
   Ehdr.e_ident[EI_DATA] =
       ELFT::TargetEndianness == support::big ? ELFDATA2MSB : ELFDATA2LSB;
template <class ELFT> void ELFWriter<ELFT>::writePhdrs() {
   for (auto &Seg : Obj.segments())
     writePhdr(Seg);
 }
 
 template <class ELFT> void ELFWriter<ELFT>::writeShdrs() {
   // This reference serves to write the dummy section header at the begining
   // of the file. It is not used for anything else
   Elf_Shdr &Shdr =
-      *reinterpret_cast<Elf_Shdr *>(Buf.getBufferStart() + Obj.SHOff);
+      *reinterpret_cast<Elf_Shdr *>(Buf->getBufferStart() + Obj.SHOff);
   Shdr.sh_name = 0;
   Shdr.sh_type = SHT_NULL;
   Shdr.sh_flags = 0;
   Shdr.sh_addr = 0;
   Shdr.sh_offset = 0;
   // See writeEhdr for why we do this.
   uint64_t Shnum = Obj.sections().size() + 1;
   if (Shnum >= SHN_LORESERVE)
if (Sec.ParentSegment == nullptr)
         return Err;
 
   return Error::success();
 }
 
 template <class ELFT> void ELFWriter<ELFT>::writeSegmentData() {
   for (Segment &Seg : Obj.segments()) {
     size_t Size = std::min<size_t>(Seg.FileSize, Seg.getContents().size());
-    std::memcpy(Buf.getBufferStart() + Seg.Offset, Seg.getContents().data(),
+    std::memcpy(Buf->getBufferStart() + Seg.Offset, Seg.getContents().data(),
                 Size);
   }
 
   // Iterate over removed sections and overwrite their old data with zeroes.
   for (auto &Sec : Obj.removedSections()) {
     Segment *Parent = Sec.ParentSegment;
     if (Parent == nullptr || Sec.Type == SHT_NOBITS || Sec.Size == 0)
       continue;
     uint64_t Offset =
         Sec.OriginalOffset - Parent->OriginalOffset + Parent->Offset;
-    std::memset(Buf.getBufferStart() + Offset, 0, Sec.Size);
+    std::memset(Buf->getBufferStart() + Offset, 0, Sec.Size);
   }
 }
 
 template <class ELFT>
-ELFWriter<ELFT>::ELFWriter(Object &Obj, Buffer &Buf, bool WSH,
+ELFWriter<ELFT>::ELFWriter(Object &Obj, raw_ostream &Buf, bool WSH,
                            bool OnlyKeepDebug)
     : Writer(Obj, Buf), WriteSectionHeaders(WSH && Obj.HadShdrs),
       OnlyKeepDebug(OnlyKeepDebug) {}
 
 Error Object::removeSections(
     bool AllowBrokenLinks, std::function<bool(const SectionBase &)> ToRemove) {
 
   auto Iter = std::stable_partition(
template <class ELFT> Error ELFWriter<ELFT>::write() {
   // header tables can overwrite it, if covered by a segment.
   writeSegmentData();
   writeEhdr();
   writePhdrs();
   if (Error E = writeSectionData())
     return E;
   if (WriteSectionHeaders)
     writeShdrs();
-  return Buf.commit();
+
+  Out.write(Buf->getBufferStart(), Buf->getBufferSize());
+  Out.flush();
+  return Error::success();
 }
 
 static Error removeUnneededSections(Object &Obj) {
   // We can remove an empty symbol table from non-relocatable objects.
   // Relocatable objects typically have relocation sections whose
   // sh_link field points to .symtab, so we can't remove .symtab
   // even if it is empty.
   if (Obj.isRelocatable() || Obj.SymbolTable == nullptr ||
template <class ELFT> Error ELFWriter<ELFT>::finalize() {
   for (SectionBase &Sec : Obj.sections()) {
     Sec.HeaderOffset = Offset;
     Offset += sizeof(Elf_Shdr);
     if (WriteSectionHeaders)
       Sec.NameIndex = Obj.SectionNames->findIndex(Sec.Name);
     Sec.finalize();
   }
 
-  if (Error E = Buf.allocate(totalSize()))
-    return E;
-  SecWriter = std::make_unique<ELFSectionWriter<ELFT>>(Buf);
+  Buf = WritableMemoryBuffer::getNewMemBuffer(totalSize());
+  if (!Buf)
+    return createStringError(llvm::errc::not_enough_memory, Twine(totalSize()));
+  SecWriter = std::make_unique<ELFSectionWriter<ELFT>>(*Buf);
   return Error::success();
 }
 
 Error BinaryWriter::write() {
   for (const SectionBase &Sec : Obj.allocSections())
     if (Error Err = Sec.accept(*SecWriter))
       return Err;
 
-  return Buf.commit();
+  // TODO: Implement direct writing to the output stream
+  // TODO: (without intermediate memory buffer Buf).
+  Out.write(Buf->getBufferStart(), Buf->getBufferSize());
+  Out.flush();
+  return Error::success();
 }
 
 Error BinaryWriter::finalize() {
   // Compute the section LMA based on its sh_offset and the containing segment's
   // p_offset and p_paddr. Also compute the minimum LMA of all non-empty
   // sections as MinAddr. In the output, the contents between address 0 and
   // MinAddr will be skipped.
   uint64_t MinAddr = UINT64_MAX;
Error BinaryWriter::finalize() {
   // its last non-empty section, to match GNU objcopy's behaviour.
   TotalSize = 0;
   for (SectionBase &Sec : Obj.allocSections())
     if (Sec.Type != SHT_NOBITS && Sec.Size > 0) {
       Sec.Offset = Sec.Addr - MinAddr;
       TotalSize = std::max(TotalSize, Sec.Offset + Sec.Size);
     }
 
-  if (Error E = Buf.allocate(TotalSize))
-    return E;
-  SecWriter = std::make_unique<BinarySectionWriter>(Buf);
+  Buf = WritableMemoryBuffer::getNewMemBuffer(TotalSize);
+  if (!Buf)
+    return createStringError(llvm::errc::not_enough_memory, Twine(TotalSize));
+  SecWriter = std::make_unique<BinarySectionWriter>(*Buf);
   return Error::success();
 }
 
 bool IHexWriter::SectionCompare::operator()(const SectionBase *Lhs,
                                             const SectionBase *Rhs) const {
   return (sectionPhysicalAddr(Lhs) & 0xFFFFFFFFU) <
          (sectionPhysicalAddr(Rhs) & 0xFFFFFFFFU);
 }
 

 
 uint64_t IHexWriter::writeEndOfFileRecord(uint8_t *Buf) {
   IHexLineData HexData = IHexRecord::getLine(IHexRecord::EndOfFile, 0, {});
   memcpy(Buf, HexData.data(), HexData.size());
   return HexData.size();
 }
 
 Error IHexWriter::write() {
-  IHexSectionWriter Writer(Buf);
+  IHexSectionWriter Writer(*Buf);
   // Write sections.
   for (const SectionBase *Sec : Sections)
     if (Error Err = Sec->accept(Writer))
       return Err;
 
   uint64_t Offset = Writer.getBufferOffset();
   // Write entry point address.
-  Offset += writeEntryPointRecord(Buf.getBufferStart() + Offset);
+  Offset += writeEntryPointRecord(
+      reinterpret_cast<uint8_t *>(Buf->getBufferStart()) + Offset);
   // Write EOF.
-  Offset += writeEndOfFileRecord(Buf.getBufferStart() + Offset);
+  Offset += writeEndOfFileRecord(
+      reinterpret_cast<uint8_t *>(Buf->getBufferStart()) + Offset);
   assert(Offset == TotalSize);
-  return Buf.commit();
+
+  Out.write(Buf->getBufferStart(), Buf->getBufferSize());
+  Out.flush();
+  return Error::success();
 }
 
 Error IHexWriter::checkSection(const SectionBase &Sec) {
   uint64_t Addr = sectionPhysicalAddr(&Sec);
   if (addressOverflows32bit(Addr) || addressOverflows32bit(Addr + Sec.Size - 1))
     return createStringError(
         errc::invalid_argument,
         "Section '%s' address range [0x%llx, 0x%llx] is not 32 bit",
Error IHexWriter::finalize() {
 
   for (const SectionBase &Sec : Obj.sections())
     if (ShouldWrite(Sec) && (!UseSegments || IsInPtLoad(Sec))) {
       if (Error E = checkSection(Sec))
         return E;
       Sections.insert(&Sec);
     }
 
-  IHexSectionWriterBase LengthCalc(Buf);
+  std::unique_ptr<WritableMemoryBuffer> EmptyBuffer =
+      WritableMemoryBuffer::getNewMemBuffer(0);
+  if (!EmptyBuffer)
+    return createStringError(errc::not_enough_memory, Twine(0));
+
+  IHexSectionWriterBase LengthCalc(*EmptyBuffer);
   for (const SectionBase *Sec : Sections)
     if (Error Err = Sec->accept(LengthCalc))
       return Err;
 
   // We need space to write section records + StartAddress record
   // (if start adress is not zero) + EndOfFile record.
   TotalSize = LengthCalc.getBufferOffset() +
               (Obj.Entry ? IHexRecord::getLineLength(4) : 0) +
               IHexRecord::getLineLength(0);
-  if (Error E = Buf.allocate(TotalSize))
-    return E;
+
+  Buf = WritableMemoryBuffer::getNewMemBuffer(TotalSize);
+  if (!Buf)
+    return createStringError(llvm::errc::not_enough_memory, Twine(TotalSize));
+
   return Error::success();
 }
 
 template class ELFBuilder<ELF64LE>;
 template class ELFBuilder<ELF64BE>;
 template class ELFBuilder<ELF32LE>;
 template class ELFBuilder<ELF32BE>;
 
 template class ELFWriter<ELF64LE>;
 template class ELFWriter<ELF64BE>;
 template class ELFWriter<ELF32LE>;
 template class ELFWriter<ELF32BE>;
 
 } // end namespace elf
 } // end namespace objcopy
 } // end namespace llvm

llvm/tools/llvm-objcopy/MachO/MachOObjcopy.h

 //===- MachOObjcopy.h -------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TOOLS_OBJCOPY_MACHOOBJCOPY_H
 #define LLVM_TOOLS_OBJCOPY_MACHOOBJCOPY_H
 
 namespace llvm {
 class Error;
+class raw_ostream;
 
 namespace object {
 class MachOObjectFile;
 class MachOUniversalBinary;
 } // end namespace object
 
 namespace objcopy {
 struct CopyConfig;
 class Buffer;
 
 namespace macho {
 Error executeObjcopyOnBinary(const CopyConfig &Config,
-                             object::MachOObjectFile &In, Buffer &Out);
+                             object::MachOObjectFile &In, raw_ostream &Out);
 
 Error executeObjcopyOnMachOUniversalBinary(
-    CopyConfig &Config, const object::MachOUniversalBinary &In, Buffer &Out);
+    CopyConfig &Config, const object::MachOUniversalBinary &In,
+    raw_ostream &Out);
 
 } // end namespace macho
 } // end namespace objcopy
 } // end namespace llvm
 
 #endif // LLVM_TOOLS_OBJCOPY_MACHOOBJCOPY_H

llvm/tools/llvm-objcopy/MachO/MachOObjcopy.cpp

 #include "MachOReader.h"
 #include "MachOWriter.h"
 #include "llvm/ADT/DenseSet.h"
 #include "llvm/Object/ArchiveWriter.h"
 #include "llvm/Object/MachOUniversal.h"
 #include "llvm/Object/MachOUniversalWriter.h"
 #include "llvm/Support/Errc.h"
 #include "llvm/Support/Error.h"
+#include "llvm/Support/FileOutputBuffer.h"
+#include "llvm/Support/SmallVectorMemoryBuffer.h"
 
 namespace llvm {
 namespace objcopy {
 namespace macho {
 
 using namespace object;
 using SectionPred = std::function<bool(const std::unique_ptr<Section> &Sec)>;
 using LoadCommandPred = std::function<bool(const LoadCommand &LC)>;
static Error handleArgs(const CopyConfig &Config, Object &Obj) {
 
   if (Error E = processLoadCommands(Config, Obj))
     return E;
 
   return Error::success();
 }
 
 Error executeObjcopyOnBinary(const CopyConfig &Config,
-                             object::MachOObjectFile &In, Buffer &Out) {
+                             object::MachOObjectFile &In, raw_ostream &Out) {
   MachOReader Reader(In);
   Expected<std::unique_ptr<Object>> O = Reader.create();
   if (!O)
     return createFileError(Config.InputFilename, O.takeError());
 
   if (Error E = handleArgs(Config, **O))
     return createFileError(Config.InputFilename, std::move(E));
 
Error executeObjcopyOnBinary(const CopyConfig &Config,
   MachOWriter Writer(**O, In.is64Bit(), In.isLittleEndian(), PageSize, Out);
   if (auto E = Writer.finalize())
     return E;
   return Writer.write();
 }
 
 Error executeObjcopyOnMachOUniversalBinary(CopyConfig &Config,
                                            const MachOUniversalBinary &In,
-                                           Buffer &Out) {
+                                           raw_ostream &Out) {
   SmallVector<OwningBinary<Binary>, 2> Binaries;
   SmallVector<Slice, 2> Slices;
   for (const auto &O : In.objects()) {
     Expected<std::unique_ptr<Archive>> ArOrErr = O.getAsArchive();
     if (ArOrErr) {
       Expected<std::vector<NewArchiveMember>> NewArchiveMembersOrErr =
           createNewArchiveMembers(Config, **ArOrErr);
       if (!NewArchiveMembersOrErr)
if (!ObjOrErr) {
       consumeError(ObjOrErr.takeError());
       return createStringError(std::errc::invalid_argument,
                                "slice for '%s' of the universal Mach-O binary "
                                "'%s' is not a Mach-O object or an archive",
                                O.getArchFlagName().c_str(),
                                Config.InputFilename.str().c_str());
     }
     std::string ArchFlagName = O.getArchFlagName();
-    MemBuffer MB(ArchFlagName);
-    if (Error E = executeObjcopyOnBinary(Config, **ObjOrErr, MB))
+
+    SmallVector<char, 0> Buffer;
+    raw_svector_ostream MemStream(Buffer);
+
+    if (Error E = executeObjcopyOnBinary(Config, **ObjOrErr, MemStream))
       return E;
-    std::unique_ptr<WritableMemoryBuffer> OutputBuffer =
-        MB.releaseMemoryBuffer();
-    Expected<std::unique_ptr<Binary>> BinaryOrErr =
-        object::createBinary(*OutputBuffer);
+
+    std::unique_ptr<MemoryBuffer> MB =
+        std::make_unique<SmallVectorMemoryBuffer>(std::move(Buffer),
+                                                  ArchFlagName);
+    Expected<std::unique_ptr<Binary>> BinaryOrErr = object::createBinary(*MB);
     if (!BinaryOrErr)
       return BinaryOrErr.takeError();
-    Binaries.emplace_back(std::move(*BinaryOrErr), std::move(OutputBuffer));
+    Binaries.emplace_back(std::move(*BinaryOrErr), std::move(MB));
     Slices.emplace_back(*cast<MachOObjectFile>(Binaries.back().getBinary()),
                         O.getAlign());
   }
-  Expected<std::unique_ptr<MemoryBuffer>> B =
-      writeUniversalBinaryToBuffer(Slices);
-  if (!B)
-    return B.takeError();
-  if (Error E = Out.allocate((*B)->getBufferSize()))
-    return E;
-  memcpy(Out.getBufferStart(), (*B)->getBufferStart(), (*B)->getBufferSize());
-  return Out.commit();
+
+  if (Error Err = writeUniversalBinaryToStream(Slices, Out))
+    return Err;
+
+  Out.flush();
+  return Error::success();
 }
 
 } // end namespace macho
 } // end namespace objcopy
 } // end namespace llvm

llvm/tools/llvm-objcopy/MachO/MachOWriter.h

 //===- MachOWriter.h --------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
-#include "../Buffer.h"
 #include "MachOLayoutBuilder.h"
 #include "MachOObjcopy.h"
 #include "Object.h"
 #include "llvm/BinaryFormat/MachO.h"
 #include "llvm/Object/MachO.h"
 
 namespace llvm {
 class Error;
 
 namespace objcopy {
 namespace macho {
 
 class MachOWriter {
   Object &O;
   bool Is64Bit;
   bool IsLittleEndian;
   uint64_t PageSize;
-  Buffer &B;
+  std::unique_ptr<WritableMemoryBuffer> Buf;
+  raw_ostream &Out;
   MachOLayoutBuilder LayoutBuilder;
 
   size_t headerSize() const;
   size_t loadCommandsSize() const;
   size_t symTableSize() const;
   size_t strTableSize() const;
 
   void writeHeader();
class MachOWriter {
   void writeLinkData(Optional<size_t> LCIndex, const LinkData &LD);
   void writeCodeSignatureData();
   void writeDataInCodeData();
   void writeFunctionStartsData();
   void writeTail();
 
 public:
   MachOWriter(Object &O, bool Is64Bit, bool IsLittleEndian, uint64_t PageSize,
-              Buffer &B)
+              raw_ostream &Out)
       : O(O), Is64Bit(Is64Bit), IsLittleEndian(IsLittleEndian),
-        PageSize(PageSize), B(B), LayoutBuilder(O, Is64Bit, PageSize) {}
+        PageSize(PageSize), Out(Out), LayoutBuilder(O, Is64Bit, PageSize) {}
 
   size_t totalSize() const;
   Error finalize();
   Error write();
 };
 
 } // end namespace macho
 } // end namespace objcopy
 } // end namespace llvm

llvm/tools/llvm-objcopy/MachO/MachOWriter.cpp

void MachOWriter::writeHeader() {
   Header.flags = O.Header.Flags;
   Header.reserved = O.Header.Reserved;
 
   if (IsLittleEndian != sys::IsLittleEndianHost)
     MachO::swapStruct(Header);
 
   auto HeaderSize =
       Is64Bit ? sizeof(MachO::mach_header_64) : sizeof(MachO::mach_header);
-  memcpy(B.getBufferStart(), &Header, HeaderSize);
+  memcpy(Buf->getBufferStart(), &Header, HeaderSize);
 }
 
 void MachOWriter::writeLoadCommands() {
-  uint8_t *Begin = B.getBufferStart() + headerSize();
+  uint8_t *Begin =
+      reinterpret_cast<uint8_t *>(Buf->getBufferStart()) + headerSize();
   for (const LoadCommand &LC : O.LoadCommands) {
     // Construct a load command.
     MachO::macho_load_command MLC = LC.MachOLoadCommand;
     switch (MLC.load_command_data.cmd) {
     case MachO::LC_SEGMENT:
       if (IsLittleEndian != sys::IsLittleEndianHost)
         MachO::swapStruct(MLC.segment_command_data);
       memcpy(Begin, &MLC.segment_command_data, sizeof(MachO::segment_command));
for (const std::unique_ptr<Section> &Sec : LC.Sections) {
         assert((Sec->Offset == 0) && "Skipped section's offset must be zero");
         assert((Sec->isVirtualSection() || Sec->Size == 0) &&
                "Non-zero-fill sections with zero offset must have zero size");
         continue;
       }
 
       assert(Sec->Offset && "Section offset can not be zero");
       assert((Sec->Size == Sec->Content.size()) && "Incorrect section size");
-      memcpy(B.getBufferStart() + Sec->Offset, Sec->Content.data(),
+      memcpy(Buf->getBufferStart() + Sec->Offset, Sec->Content.data(),
              Sec->Content.size());
       for (size_t Index = 0; Index < Sec->Relocations.size(); ++Index) {
         RelocationInfo RelocInfo = Sec->Relocations[Index];
         if (!RelocInfo.Scattered) {
           const uint32_t SymbolNum = RelocInfo.Extern
                                          ? (*RelocInfo.Symbol)->Index
                                          : (*RelocInfo.Sec)->Index;
           RelocInfo.setPlainRelocationSymbolNum(SymbolNum, IsLittleEndian);
         }
         if (IsLittleEndian != sys::IsLittleEndianHost)
           MachO::swapStruct(
               reinterpret_cast<MachO::any_relocation_info &>(RelocInfo.Info));
-        memcpy(B.getBufferStart() + Sec->RelOff +
+        memcpy(Buf->getBufferStart() + Sec->RelOff +
                    Index * sizeof(MachO::any_relocation_info),
                &RelocInfo.Info, sizeof(RelocInfo.Info));
       }
     }
 }
 
 template <typename NListType>
 void writeNListEntry(const SymbolEntry &SE, bool IsLittleEndian, char *&Out,

 
 void MachOWriter::writeStringTable() {
   if (!O.SymTabCommandIndex)
     return;
   const MachO::symtab_command &SymTabCommand =
       O.LoadCommands[*O.SymTabCommandIndex]
           .MachOLoadCommand.symtab_command_data;
 
-  uint8_t *StrTable = (uint8_t *)B.getBufferStart() + SymTabCommand.stroff;
+  uint8_t *StrTable = (uint8_t *)Buf->getBufferStart() + SymTabCommand.stroff;
   LayoutBuilder.getStringTableBuilder().write(StrTable);
 }
 
 void MachOWriter::writeSymbolTable() {
   if (!O.SymTabCommandIndex)
     return;
   const MachO::symtab_command &SymTabCommand =
       O.LoadCommands[*O.SymTabCommandIndex]
           .MachOLoadCommand.symtab_command_data;
 
-  char *SymTable = (char *)B.getBufferStart() + SymTabCommand.symoff;
+  char *SymTable = (char *)Buf->getBufferStart() + SymTabCommand.symoff;
   for (auto Iter = O.SymTable.Symbols.begin(), End = O.SymTable.Symbols.end();
        Iter != End; Iter++) {
     SymbolEntry *Sym = Iter->get();
     uint32_t Nstrx = LayoutBuilder.getStringTableBuilder().getOffset(Sym->Name);
 
     if (Is64Bit)
       writeNListEntry<MachO::nlist_64>(*Sym, IsLittleEndian, SymTable, Nstrx);
     else
       writeNListEntry<MachO::nlist>(*Sym, IsLittleEndian, SymTable, Nstrx);
   }
 }
 
 void MachOWriter::writeRebaseInfo() {
   if (!O.DyLdInfoCommandIndex)
     return;
   const MachO::dyld_info_command &DyLdInfoCommand =
       O.LoadCommands[*O.DyLdInfoCommandIndex]
           .MachOLoadCommand.dyld_info_command_data;
-  char *Out = (char *)B.getBufferStart() + DyLdInfoCommand.rebase_off;
+  char *Out = (char *)Buf->getBufferStart() + DyLdInfoCommand.rebase_off;
   assert((DyLdInfoCommand.rebase_size == O.Rebases.Opcodes.size()) &&
          "Incorrect rebase opcodes size");
   memcpy(Out, O.Rebases.Opcodes.data(), O.Rebases.Opcodes.size());
 }
 
 void MachOWriter::writeBindInfo() {
   if (!O.DyLdInfoCommandIndex)
     return;
   const MachO::dyld_info_command &DyLdInfoCommand =
       O.LoadCommands[*O.DyLdInfoCommandIndex]
           .MachOLoadCommand.dyld_info_command_data;
-  char *Out = (char *)B.getBufferStart() + DyLdInfoCommand.bind_off;
+  char *Out = (char *)Buf->getBufferStart() + DyLdInfoCommand.bind_off;
   assert((DyLdInfoCommand.bind_size == O.Binds.Opcodes.size()) &&
          "Incorrect bind opcodes size");
   memcpy(Out, O.Binds.Opcodes.data(), O.Binds.Opcodes.size());
 }
 
 void MachOWriter::writeWeakBindInfo() {
   if (!O.DyLdInfoCommandIndex)
     return;
   const MachO::dyld_info_command &DyLdInfoCommand =
       O.LoadCommands[*O.DyLdInfoCommandIndex]
           .MachOLoadCommand.dyld_info_command_data;
-  char *Out = (char *)B.getBufferStart() + DyLdInfoCommand.weak_bind_off;
+  char *Out = (char *)Buf->getBufferStart() + DyLdInfoCommand.weak_bind_off;
   assert((DyLdInfoCommand.weak_bind_size == O.WeakBinds.Opcodes.size()) &&
          "Incorrect weak bind opcodes size");
   memcpy(Out, O.WeakBinds.Opcodes.data(), O.WeakBinds.Opcodes.size());
 }
 
 void MachOWriter::writeLazyBindInfo() {
   if (!O.DyLdInfoCommandIndex)
     return;
   const MachO::dyld_info_command &DyLdInfoCommand =
       O.LoadCommands[*O.DyLdInfoCommandIndex]
           .MachOLoadCommand.dyld_info_command_data;
-  char *Out = (char *)B.getBufferStart() + DyLdInfoCommand.lazy_bind_off;
+  char *Out = (char *)Buf->getBufferStart() + DyLdInfoCommand.lazy_bind_off;
   assert((DyLdInfoCommand.lazy_bind_size == O.LazyBinds.Opcodes.size()) &&
          "Incorrect lazy bind opcodes size");
   memcpy(Out, O.LazyBinds.Opcodes.data(), O.LazyBinds.Opcodes.size());
 }
 
 void MachOWriter::writeExportInfo() {
   if (!O.DyLdInfoCommandIndex)
     return;
   const MachO::dyld_info_command &DyLdInfoCommand =
       O.LoadCommands[*O.DyLdInfoCommandIndex]
           .MachOLoadCommand.dyld_info_command_data;
-  char *Out = (char *)B.getBufferStart() + DyLdInfoCommand.export_off;
+  char *Out = (char *)Buf->getBufferStart() + DyLdInfoCommand.export_off;
   assert((DyLdInfoCommand.export_size == O.Exports.Trie.size()) &&
          "Incorrect export trie size");
   memcpy(Out, O.Exports.Trie.data(), O.Exports.Trie.size());
 }
 
 void MachOWriter::writeIndirectSymbolTable() {
   if (!O.DySymTabCommandIndex)
     return;
 
   const MachO::dysymtab_command &DySymTabCommand =
       O.LoadCommands[*O.DySymTabCommandIndex]
           .MachOLoadCommand.dysymtab_command_data;
 
   uint32_t *Out =
-      (uint32_t *)(B.getBufferStart() + DySymTabCommand.indirectsymoff);
+      (uint32_t *)(Buf->getBufferStart() + DySymTabCommand.indirectsymoff);
   for (const IndirectSymbolEntry &Sym : O.IndirectSymTable.Symbols) {
     uint32_t Entry = (Sym.Symbol) ? (*Sym.Symbol)->Index : Sym.OriginalIndex;
     if (IsLittleEndian != sys::IsLittleEndianHost)
       sys::swapByteOrder(Entry);
     *Out++ = Entry;
   }
 }
 
 void MachOWriter::writeLinkData(Optional<size_t> LCIndex, const LinkData &LD) {
   if (!LCIndex)
     return;
   const MachO::linkedit_data_command &LinkEditDataCommand =
       O.LoadCommands[*LCIndex].MachOLoadCommand.linkedit_data_command_data;
-  char *Out = (char *)B.getBufferStart() + LinkEditDataCommand.dataoff;
+  char *Out = (char *)Buf->getBufferStart() + LinkEditDataCommand.dataoff;
   assert((LinkEditDataCommand.datasize == LD.Data.size()) &&
          "Incorrect data size");
   memcpy(Out, LD.Data.data(), LD.Data.size());
 }
 
 void MachOWriter::writeCodeSignatureData() {
   return writeLinkData(O.CodeSignatureCommandIndex, O.CodeSignature);
 }
void MachOWriter::writeTail() {
 
   for (auto WriteOp : Queue)
     (this->*WriteOp.second)();
 }
 
 Error MachOWriter::finalize() { return LayoutBuilder.layout(); }
 
 Error MachOWriter::write() {
-  if (Error E = B.allocate(totalSize()))
-    return E;
-  memset(B.getBufferStart(), 0, totalSize());
+  Buf = WritableMemoryBuffer::getNewMemBuffer(totalSize());
+  if (!Buf)
+    return createStringError(llvm::errc::not_enough_memory, Twine(totalSize()));
+  memset(Buf->getBufferStart(), 0, totalSize());
   writeHeader();
   writeLoadCommands();
   writeSections();
   writeTail();
-  return B.commit();
+
+  // TODO: Implement direct writing to the output stream
+  // TODO: (without intermediate memory buffer Buf).
+  Out.write(Buf->getBufferStart(), Buf->getBufferSize());
+  Out.flush();
+  return Error::success();
 }
 
 } // end namespace macho
 } // end namespace objcopy
 } // end namespace llvm

llvm/tools/llvm-objcopy/Util.h

+//===- Util.h ---------------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_TOOLS_LLVM_OBJCOPY_UTIL_H
+#define LLVM_TOOLS_LLVM_OBJCOPY_UTIL_H
+
+#include "llvm/Support/Error.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace llvm {
+namespace objcopy {
+
+inline Error writeToFile(StringRef OutputFileName,
+                         std::function<Error(raw_ostream &)> Write) {
+  if (OutputFileName == "-")
+    return Write(outs());
+
+  if (OutputFileName == "/dev/null") {
+    raw_null_ostream Out;
+    return Write(Out);
+  }
+
+  unsigned Mode = sys::fs::all_read | sys::fs::all_write | sys::fs::all_exe;
+  Expected<sys::fs::TempFile> Temp =
+      sys::fs::TempFile::create(OutputFileName + ".temp-objcopy-%%%%%%", Mode);
+  if (!Temp)
+    return createFileError(OutputFileName, Temp.takeError());
+
+  raw_fd_ostream Out(Temp->FD, false);
+
+  if (Error E = Write(Out)) {
+    if (Error DiscardError = Temp->discard())
+      return joinErrors(std::move(E), std::move(DiscardError));
+    return E;
+  }
+
+  return Temp->keep(OutputFileName);
+}
+
+} // end namespace objcopy
+} // end namespace llvm
+
+#endif // LLVM_TOOLS_LLVM_OBJCOPY_UTIL_H

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

 //===- llvm-objcopy.cpp ---------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
-#include "Buffer.h"
 #include "COFF/COFFObjcopy.h"
 #include "CopyConfig.h"
 #include "ELF/ELFObjcopy.h"
 #include "MachO/MachOObjcopy.h"
+#include "Util.h"
 #include "wasm/WasmObjcopy.h"
 
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Twine.h"
+#include "llvm/BinaryFormat/ELF.h"
 #include "llvm/Object/Archive.h"
 #include "llvm/Object/ArchiveWriter.h"
 #include "llvm/Object/Binary.h"
 #include "llvm/Object/COFF.h"
 #include "llvm/Object/ELFObjectFile.h"
 #include "llvm/Object/ELFTypes.h"
 #include "llvm/Object/Error.h"
 #include "llvm/Object/MachO.h"
 #include "llvm/Object/MachOUniversal.h"
 #include "llvm/Object/Wasm.h"
 #include "llvm/Option/Arg.h"
 #include "llvm/Option/ArgList.h"
 #include "llvm/Option/Option.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/CommandLine.h"
+#include "llvm/Support/Errc.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/ErrorOr.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/InitLLVM.h"
 #include "llvm/Support/Memory.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/Process.h"
+#include "llvm/Support/SmallVectorMemoryBuffer.h"
 #include "llvm/Support/StringSaver.h"
 #include "llvm/Support/WithColor.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
 #include <cassert>
 #include <cstdlib>
 #include <memory>
 #include <string>
static Expected<DriverConfig> getDriverConfig(ArrayRef<const char *> Args) {
   else if (Is("strip"))
     return parseStripOptions(Args, reportWarning);
   else if (Is("install-name-tool") || Is("install_name_tool"))
     return parseInstallNameToolOptions(Args);
   else
     return parseObjcopyOptions(Args, reportWarning);
 }
 
+template <class ELFT>
+static Expected<ArrayRef<uint8_t>>
+findBuildID(const CopyConfig &Config, const object::ELFFile<ELFT> &In) {
+  auto PhdrsOrErr = In.program_headers();
+  if (auto Err = PhdrsOrErr.takeError())
+    return createFileError(Config.InputFilename, std::move(Err));
+
+  for (const auto &Phdr : *PhdrsOrErr) {
+    if (Phdr.p_type != ELF::PT_NOTE)
+      continue;
+    Error Err = Error::success();
+    for (auto Note : In.notes(Phdr, Err))
+      if (Note.getType() == ELF::NT_GNU_BUILD_ID &&
+          Note.getName() == ELF::ELF_NOTE_GNU)
+        return Note.getDesc();
+    if (Err)
+      return createFileError(Config.InputFilename, std::move(Err));
+  }
+
+  return createFileError(Config.InputFilename,
+                         createStringError(llvm::errc::invalid_argument,
+                                           "could not find build ID"));
+}
+
+static Expected<ArrayRef<uint8_t>>
+findBuildID(const CopyConfig &Config, const object::ELFObjectFileBase &In) {
+  if (auto *O = dyn_cast<object::ELFObjectFile<object::ELF32LE>>(&In))
+    return findBuildID(Config, O->getELFFile());
+  else if (auto *O = dyn_cast<object::ELFObjectFile<object::ELF64LE>>(&In))
+    return findBuildID(Config, O->getELFFile());
+  else if (auto *O = dyn_cast<object::ELFObjectFile<object::ELF32BE>>(&In))
+    return findBuildID(Config, O->getELFFile());
+  else if (auto *O = dyn_cast<object::ELFObjectFile<object::ELF64BE>>(&In))
+    return findBuildID(Config, O->getELFFile());
+
+  llvm_unreachable("Bad file format");
+}
+
+template <class... Ts>
+static Error makeStringError(std::error_code EC, const Twine &Msg,
+                             Ts &&... Args) {
+  std::string FullMsg = (EC.message() + ": " + Msg).str();
+  return createStringError(EC, FullMsg.c_str(), std::forward<Ts>(Args)...);
+}
+
+#define MODEL_8 "%%%%%%%%"
+#define MODEL_16 MODEL_8 MODEL_8
+#define MODEL_32 (MODEL_16 MODEL_16)
+
+static Error linkToBuildIdDir(const CopyConfig &Config, StringRef ToLink,
+                              StringRef Suffix,
+                              ArrayRef<uint8_t> BuildIdBytes) {
+  SmallString<128> Path = Config.BuildIdLinkDir;
+  sys::path::append(Path, llvm::toHex(BuildIdBytes[0], /*LowerCase*/ true));
+  if (auto EC = sys::fs::create_directories(Path))
+    return createFileError(
+        Path.str(),
+        makeStringError(EC, "cannot create build ID link directory"));
+
+  sys::path::append(Path,
+                    llvm::toHex(BuildIdBytes.slice(1), /*LowerCase*/ true));
+  Path += Suffix;
+  SmallString<128> TmpPath;
+  // create_hard_link races so we need to link to a temporary path but
+  // we want to make sure that we choose a filename that does not exist.
+  // By using 32 model characters we get 128-bits of entropy. It is
+  // unlikely that this string has ever existed before much less exists
+  // on this disk or in the current working directory.
+  // Additionally we prepend the original Path for debugging but also
+  // because it ensures that we're linking within a directory on the same
+  // partition on the same device which is critical. It has the added
+  // win of yet further decreasing the odds of a conflict.
+  sys::fs::createUniquePath(Twine(Path) + "-" + MODEL_32 + ".tmp", TmpPath,
+                            /*MakeAbsolute*/ false);
+  if (auto EC = sys::fs::create_hard_link(ToLink, TmpPath)) {
+    Path.push_back('\0');
+    return makeStringError(EC, "cannot link '%s' to '%s'", ToLink.data(),
+                           Path.data());
+  }
+  // We then atomically rename the link into place which will just move the
+  // link. If rename fails something is more seriously wrong so just return
+  // an error.
+  if (auto EC = sys::fs::rename(TmpPath, Path)) {
+    Path.push_back('\0');
+    return makeStringError(EC, "cannot link '%s' to '%s'", ToLink.data(),
+                           Path.data());
+  }
+  // If `Path` was already a hard-link to the same underlying file then the
+  // temp file will be left so we need to remove it. Remove will not cause
+  // an error by default if the file is already gone so just blindly remove
+  // it rather than checking.
+  if (auto EC = sys::fs::remove(TmpPath)) {
+    TmpPath.push_back('\0');
+    return makeStringError(EC, "could not remove '%s'", TmpPath.data());
+  }
+  return Error::success();
+}
+
 } // end namespace objcopy
 } // end namespace llvm
 
 using namespace llvm;
 using namespace llvm::object;
 using namespace llvm::objcopy;
 
 // For regular archives this function simply calls llvm::writeArchive,
 // For thin archives it writes the archive file itself as well as its members.
 static Error deepWriteArchive(StringRef ArcName,
                               ArrayRef<NewArchiveMember> NewMembers,
                               bool WriteSymtab, object::Archive::Kind Kind,
                               bool Deterministic, bool Thin) {
   if (Error E = writeArchive(ArcName, NewMembers, WriteSymtab, Kind,
                              Deterministic, Thin))
     return createFileError(ArcName, std::move(E));
 
   if (!Thin)
     return Error::success();
 
   for (const NewArchiveMember &Member : NewMembers) {
-    // Internally, FileBuffer will use the buffer created by
-    // FileOutputBuffer::create, for regular files (that is the case for
-    // deepWriteArchive) FileOutputBuffer::create will return OnDiskBuffer.
+    // FileOutputBuffer::create, for regular files(that is the case for
+    // deepWriteArchive) will return OnDiskBuffer.
     // OnDiskBuffer uses a temporary file and then renames it. So in reality
     // there is no inefficiency / duplicated in-memory buffers in this case. For
     // now in-memory buffers can not be completely avoided since
     // NewArchiveMember still requires them even though writeArchive does not
     // write them on disk.
-    FileBuffer FB(Member.MemberName);
-    if (Error E = FB.allocate(Member.Buf->getBufferSize()))
-      return E;
+    Expected<std::unique_ptr<FileOutputBuffer>> FB =
+        FileOutputBuffer::create(Member.MemberName, Member.Buf->getBufferSize(),
+                                 FileOutputBuffer::F_executable);
+    if (!FB)
+      return FB.takeError();
     std::copy(Member.Buf->getBufferStart(), Member.Buf->getBufferEnd(),
-              FB.getBufferStart());
-    if (Error E = FB.commit())
+              (*FB)->getBufferStart());
+    if (Error E = (*FB)->commit())
       return E;
   }
   return Error::success();
 }
 
 /// The function executeObjcopyOnIHex does the dispatch based on the format
 /// of the output specified by the command line options.
 static Error executeObjcopyOnIHex(CopyConfig &Config, MemoryBuffer &In,
-                                  Buffer &Out) {
+                                  raw_ostream &Out) {
   // TODO: support output formats other than ELF.
   if (Error E = Config.parseELFConfig())
     return E;
   return elf::executeObjcopyOnIHex(Config, In, Out);
 }
 
 /// The function executeObjcopyOnRawBinary does the dispatch based on the format
 /// of the output specified by the command line options.
 static Error executeObjcopyOnRawBinary(CopyConfig &Config, MemoryBuffer &In,
-                                       Buffer &Out) {
+                                       raw_ostream &Out) {
   switch (Config.OutputFormat) {
   case FileFormat::ELF:
   // FIXME: Currently, we call elf::executeObjcopyOnRawBinary even if the
   // output format is binary/ihex or it's not given. This behavior differs from
   // GNU objcopy. See https://bugs.llvm.org/show_bug.cgi?id=42171 for details.
   case FileFormat::Binary:
   case FileFormat::IHex:
   case FileFormat::Unspecified:
     if (Error E = Config.parseELFConfig())
       return E;
     return elf::executeObjcopyOnRawBinary(Config, In, Out);
   }
 
   llvm_unreachable("unsupported output format");
 }
 
 /// The function executeObjcopyOnBinary does the dispatch based on the format
 /// of the input binary (ELF, MachO or COFF).
 static Error executeObjcopyOnBinary(CopyConfig &Config, object::Binary &In,
-                                    Buffer &Out) {
+                                    raw_ostream &Out) {
   if (auto *ELFBinary = dyn_cast<object::ELFObjectFileBase>(&In)) {
     if (Error E = Config.parseELFConfig())
       return E;
     return elf::executeObjcopyOnBinary(Config, *ELFBinary, Out);
   } else if (auto *COFFBinary = dyn_cast<object::COFFObjectFile>(&In))
     return coff::executeObjcopyOnBinary(Config, *COFFBinary, Out);
   else if (auto *MachOBinary = dyn_cast<object::MachOObjectFile>(&In))
     return macho::executeObjcopyOnBinary(Config, *MachOBinary, Out);
for (const Archive::Child &Child : Ar.children(Err)) {
     if (!ChildNameOrErr)
       return createFileError(Ar.getFileName(), ChildNameOrErr.takeError());
 
     Expected<std::unique_ptr<Binary>> ChildOrErr = Child.getAsBinary();
     if (!ChildOrErr)
       return createFileError(Ar.getFileName() + "(" + *ChildNameOrErr + ")",
                              ChildOrErr.takeError());
 
-    MemBuffer MB(ChildNameOrErr.get());
-    if (Error E = executeObjcopyOnBinary(Config, *ChildOrErr->get(), MB))
+    SmallVector<char, 0> Buffer;
+    raw_svector_ostream MemStream(Buffer);
+
+    if (Error E = executeObjcopyOnBinary(Config, *ChildOrErr->get(), MemStream))
       return std::move(E);
 
     Expected<NewArchiveMember> Member =
         NewArchiveMember::getOldMember(Child, Config.DeterministicArchives);
     if (!Member)
       return createFileError(Ar.getFileName(), Member.takeError());
-    Member->Buf = MB.releaseMemoryBuffer();
+
+    Member->Buf = std::make_unique<SmallVectorMemoryBuffer>(
+        std::move(Buffer), ChildNameOrErr.get());
     Member->MemberName = Member->Buf->getBufferIdentifier();
     NewArchiveMembers.push_back(std::move(*Member));
   }
   if (Err)
     return createFileError(Config.InputFilename, std::move(Err));
   return std::move(NewArchiveMembers);
 }
 
static Error executeObjcopy(CopyConfig &Config) {
   sys::fs::file_status Stat;
   if (Config.InputFilename != "-") {
     if (auto EC = sys::fs::status(Config.InputFilename, Stat))
       return createFileError(Config.InputFilename, EC);
   } else {
     Stat.permissions(static_cast<sys::fs::perms>(0777));
   }
 
-  using ProcessRawFn = Error (*)(CopyConfig &, MemoryBuffer &, Buffer &);
+  using ProcessRawFn = Error (*)(CopyConfig &, MemoryBuffer &, raw_ostream &);
   ProcessRawFn ProcessRaw;
   switch (Config.InputFormat) {
   case FileFormat::Binary:
     ProcessRaw = executeObjcopyOnRawBinary;
     break;
   case FileFormat::IHex:
     ProcessRaw = executeObjcopyOnIHex;
     break;
   default:
     ProcessRaw = nullptr;
   }
 
   if (ProcessRaw) {
     auto BufOrErr = MemoryBuffer::getFileOrSTDIN(Config.InputFilename);
     if (!BufOrErr)
       return createFileError(Config.InputFilename, BufOrErr.getError());
-    FileBuffer FB(Config.OutputFilename);
-    if (Error E = ProcessRaw(Config, *BufOrErr->get(), FB))
+
+    if (Error E = writeToFile(
+            Config.OutputFilename, [&](raw_ostream &OutFile) -> Error {
+              return ProcessRaw(Config, *BufOrErr->get(), OutFile);
+            }))
       return E;
   } else {
     Expected<OwningBinary<llvm::object::Binary>> BinaryOrErr =
         createBinary(Config.InputFilename);
     if (!BinaryOrErr)
       return createFileError(Config.InputFilename, BinaryOrErr.takeError());
 
     if (Archive *Ar = dyn_cast<Archive>(BinaryOrErr.get().getBinary())) {
       if (Error E = executeObjcopyOnArchive(Config, *Ar))
         return E;
     } else {
-      FileBuffer FB(Config.OutputFilename);
-      if (Error E = executeObjcopyOnBinary(Config,
-                                           *BinaryOrErr.get().getBinary(), FB))
+      ArrayRef<uint8_t> BuildIdBytes;
+      if (auto *ELFBinary = dyn_cast<object::ELFObjectFileBase>(
+              BinaryOrErr.get().getBinary())) {
+        if (!Config.BuildIdLinkDir.empty()) {
+          auto BuildIdBytesOrErr = findBuildID(Config, *ELFBinary);
+          if (auto E = BuildIdBytesOrErr.takeError())
+            return E;
+          BuildIdBytes = *BuildIdBytesOrErr;
+
+          if (BuildIdBytes.size() < 2)
+            return createFileError(
+                Config.InputFilename,
+                createStringError(object_error::parse_failed,
+                                  "build ID is smaller than two bytes"));
+        }
+
+        if (!Config.BuildIdLinkDir.empty() && Config.BuildIdLinkInput)
+          if (Error E = linkToBuildIdDir(Config, Config.InputFilename,
+                                         Config.BuildIdLinkInput.getValue(),
+                                         BuildIdBytes))
+            return E;
+      }
+
+      if (Error E = writeToFile(
+              Config.OutputFilename, [&](raw_ostream &OutFile) -> Error {
+                return executeObjcopyOnBinary(
+                    Config, *BinaryOrErr.get().getBinary(), OutFile);
+              }))
         return E;
+
+      if (auto *ELFBinary = dyn_cast<object::ELFObjectFileBase>(
+              BinaryOrErr.get().getBinary())) {
+        if (!Config.BuildIdLinkDir.empty() && Config.BuildIdLinkOutput)
+          if (Error E = linkToBuildIdDir(Config, Config.OutputFilename,
+                                         Config.BuildIdLinkOutput.getValue(),
+                                         BuildIdBytes))
+            return createFileError(Config.OutputFilename, std::move(E));
+      }
     }
   }
 
   if (Error E =
           restoreStatOnFile(Config.OutputFilename, Stat, Config.PreserveDates))
     return E;
 
   if (!Config.SplitDWO.empty()) {

llvm/tools/llvm-objcopy/wasm/WasmObjcopy.h

 //===- WasmObjcopy.h -------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TOOLS_LLVM_OBJCOPY_WASM_WASMOBJCOPY_H
 #define LLVM_TOOLS_LLVM_OBJCOPY_WASM_WASMOBJCOPY_H
 
 namespace llvm {
 class Error;
+class raw_ostream;
 
 namespace object {
 class WasmObjectFile;
 } // end namespace object
 
 namespace objcopy {
 struct CopyConfig;
-class Buffer;
 
 namespace wasm {
 Error executeObjcopyOnBinary(const CopyConfig &Config,
-                             object::WasmObjectFile &In, Buffer &Out);
+                             object::WasmObjectFile &In, raw_ostream &Out);
 
 } // end namespace wasm
 } // end namespace objcopy
 } // end namespace llvm
 
 #endif // LLVM_TOOLS_LLVM_OBJCOPY_WASM_WASMOBJCOPY_H

llvm/tools/llvm-objcopy/wasm/WasmObjcopy.cpp

 //===- WasmObjcopy.cpp ----------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "WasmObjcopy.h"
-#include "Buffer.h"
 #include "CopyConfig.h"
 #include "Object.h"
 #include "Reader.h"
 #include "Writer.h"
 #include "llvm/Support/Errc.h"
+#include "llvm/Support/FileOutputBuffer.h"
 
 namespace llvm {
 namespace objcopy {
 namespace wasm {
 
 using namespace object;
 
 static Error dumpSectionToFile(StringRef SecName, StringRef Filename,
if (!Config.AddGnuDebugLink.empty() || !Config.BuildIdLinkDir.empty() ||
     return createStringError(
         llvm::errc::invalid_argument,
         "only add-section, dump-section, and remove-section are supported");
   }
   return Error::success();
 }
 
 Error executeObjcopyOnBinary(const CopyConfig &Config,
-                             object::WasmObjectFile &In, Buffer &Out) {
+                             object::WasmObjectFile &In, raw_ostream &Out) {
   Reader TheReader(In);
   Expected<std::unique_ptr<Object>> ObjOrErr = TheReader.create();
   if (!ObjOrErr)
     return createFileError(Config.InputFilename, ObjOrErr.takeError());
   Object *Obj = ObjOrErr->get();
   assert(Obj && "Unable to deserialize Wasm object");
   if (Error E = handleArgs(Config, *Obj))
     return E;

llvm/tools/llvm-objcopy/wasm/Writer.h

 //===- Writer.h -------------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TOOLS_LLVM_OBJCOPY_WASM_WRITER_H
 #define LLVM_TOOLS_LLVM_OBJCOPY_WASM_WRITER_H
 
-#include "Buffer.h"
 #include "Object.h"
 #include <cstdint>
 #include <vector>
 
 namespace llvm {
 namespace objcopy {
 namespace wasm {
 
 class Writer {
 public:
-  Writer(Object &Obj, Buffer &Buf) : Obj(Obj), Buf(Buf) {}
+  Writer(Object &Obj, raw_ostream &Out) : Obj(Obj), Out(Out) {}
   Error write();
 
 private:
   using SectionHeader = SmallVector<char, 8>;
   Object &Obj;
-  Buffer &Buf;
+  raw_ostream &Out;
   std::vector<SectionHeader> SectionHeaders;
 
   /// Generate a wasm section section header for S.
   /// The header consists of
   /// * A one-byte section ID (aka the section type).
   /// * The size of the section contents, encoded as ULEB128.
   /// * If the section is a custom section (type 0) it also has a name, which is
   ///   encoded as a length-prefixed string. The encoded section size *includes*

llvm/tools/llvm-objcopy/wasm/Writer.cpp

 //===- Writer.cpp ---------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "Writer.h"
 #include "llvm/BinaryFormat/Wasm.h"
 #include "llvm/Support/Endian.h"
+#include "llvm/Support/Errc.h"
 #include "llvm/Support/LEB128.h"
 #include "llvm/Support/raw_ostream.h"
 
 namespace llvm {
 namespace objcopy {
 namespace wasm {
 
 using namespace object;
for (const Section &S : Obj.Sections) {
     size_t SectionSize;
     SectionHeaders.push_back(createSectionHeader(S, SectionSize));
     ObjectSize += SectionSize;
   }
   return ObjectSize;
 }
 
 Error Writer::write() {
-  size_t FileSize = finalize();
-  if (Error E = Buf.allocate(FileSize))
-    return E;
+  size_t TotalSize = finalize();
+  std::unique_ptr<WritableMemoryBuffer> Buf =
+      WritableMemoryBuffer::getNewMemBuffer(TotalSize);
+  if (!Buf)
+    return createStringError(llvm::errc::not_enough_memory, Twine(TotalSize));
 
   // Write the header.
-  uint8_t *Ptr = Buf.getBufferStart();
+  uint8_t *Ptr = reinterpret_cast<uint8_t *>(Buf->getBufferStart());
   Ptr = std::copy(Obj.Header.Magic.begin(), Obj.Header.Magic.end(), Ptr);
   support::endian::write32le(Ptr, Obj.Header.Version);
   Ptr += sizeof(Obj.Header.Version);
 
   // Write each section.
   for (size_t I = 0, S = SectionHeaders.size(); I < S; ++I) {
     Ptr = std::copy(SectionHeaders[I].begin(), SectionHeaders[I].end(), Ptr);
     ArrayRef<uint8_t> Contents = Obj.Sections[I].Contents;
     Ptr = std::copy(Contents.begin(), Contents.end(), Ptr);
   }
-  return Buf.commit();
+
+  // TODO: Implement direct writing to the output stream
+  // TODO: (without intermediate memory buffer Buf).
+  Out.write(Buf->getBufferStart(), Buf->getBufferSize());
+  Out.flush();
+  return Error::success();
 }
 
 } // end namespace wasm
 } // end namespace objcopy
 } // end namespace llvm