Commit Graph

94 Commits

Author SHA1 Message Date
Andrew Kelley d2cdfb9490 stage2: add 4 new linker flags for WebAssembly
--import-memory          import memory from the environment
--initial-memory=[bytes] initial size of the linear memory
--max-memory=[bytes]     maximum size of the linear memory
--global-base=[addr]     where to start to place global data

See #8633
2021-11-09 14:29:20 -07:00
Ryan Liptak e97feb96e4 Replace ArrayList.init/ensureTotalCapacity pairs with initCapacity
Because ArrayList.initCapacity uses 'precise' capacity allocation, this should save memory on average, and definitely will save memory in cases where ArrayList is used where a regular allocated slice could have also be used.
2021-11-04 14:54:25 -04:00
Kenta Iwasaki 2cdffc97f0 zig: expose linker options and include '-z notext'
Add an option to allow the '-z notext' option to be passed to the linker
via. the compiler frontend, which is a flag that tells the linker that
relocations in read-only sections are permitted. Certain targets such as
Solana BPF rely on this flag.

Expose all linker options i.e. '-z nodelete', '-z now', '-z relro' in
the compiler frontend. Usage documentation has been updated accordingly.

Expose the '-z notext' flag in the standard library build runner.
2021-10-29 19:18:44 -04:00
Andrew Kelley 30d01c8fea Merge pull request #9874 from leecannon/frame_pointer
Make omiting frame pointer independent of build mode
2021-10-25 19:15:17 -04:00
Jakub Konka d0dceae736 macho: dump linker's state as JSON
Each element of the output JSON has the VM address of the generated
binary nondecreasing (some elements might occupy the same VM address
for example the atom and the relocation might coincide in the address
space).

The generated JSON can be inspected manually or via a preview tool
`zig-snapshots` that I am currently working on and will allow the user
to inspect interactively the state of the linker together with the
positioning of sections, symbols, atoms and relocations within each
snapshot state, and in the future, between snapshots too. This should
allow for quicker debugging of the linker which is nontrivial when
run in the incremental mode.

Note that the state will only be dumped if the compiler is built with
`-Dlink-snapshot` flag on, and then the compiler is passed `--debug-link-snapshot`
flag upon compiling a source/project.
2021-10-22 12:50:25 +02:00
Jakub Konka 372e9709ad macho: fix LLVM codepaths in self-hosted linker
* do not add linkage scope to aliased exported symbols - this is
  not respected on macOS
* special-case `MachO.openPath` in `link.File.openPath` as on macOS
  we always link with zld
* redirect to `MachO.flushObject` when linking relocatable objects
  in MachO linker whereas move the entire linking logic into
  `MachO.flushModule`
2021-10-19 20:39:42 +02:00
Lee Cannon b15d6b2a34 Add build.zig and command line flags 2021-10-16 21:55:51 +01:00
Andrew Kelley 6115cf2240 migrate from std.Target.current to @import("builtin").target
closes #9388
closes #9321
2021-10-04 23:48:55 -07:00
Andrew Kelley ef7fa76001 stage2: enable building freestanding libc with LLVM backend
* LLVM backend: respect `sub_path` just like the other stage2 backends
   do.
 * Compilation has some new logic to only emit work queue jobs for
   building stuff when it believes itself to be capable. The linker
   backends no longer have duplicate logic; instead they respect the
   optional bit on the respective asset.
2021-09-24 01:00:15 -07:00
Andrew Kelley 5913140b6b stage2: free Sema's arena after generating machine code
Previously, linker backends or machine code backends were able to hold
on to references to inside Sema's temporary arena. However there can
be large objects stored there that we want to free after machine code is
generated.

The primary change in this commit is to use a temporary arena for Sema
of function bodies that gets freed after machine code backend finishes
handling `updateFunc` (at the same time that Air and Liveness get freed).

The other changes in this commit are fixing issues that fell out from
the primary change.

 * The C linker backend is rewritten to handle updateDecl and updateFunc
   separately. Also, all Decl updates get access to typedefs and
   fwd_decls, not only functions.
 * The C linker backend is updated to the new API that does not depend
   on allocateDeclIndexes and does not have to handle garbage collected
   decls.
 * The C linker backend uses an arena for Type/Value objects that
   `typedefs` references. These can be garbage collected every so often
   after flush(), however that garbage collection code is not
   implemented at this time. It will be pretty simple, just allocate a
   new arena, copy all the Type objects to it, update the keys of the
   hash map, free the old arena.
 * Sema: fix a handful of instances of not copying Type/Value objects
   from the temporary arena into the appropriate Decl arena.
 * Type: fix some function types not reporting hasCodeGenBits()
   correctly.
2021-09-21 15:23:29 -07:00
Ryan Liptak 59f5053bed Update all ensureCapacity calls to the relevant non-deprecated version 2021-09-19 13:52:56 +02:00
Alex Rønne Petersen eb5e4ac495 link: Recognize -z origin|noexecstack|now|relro linker args. 2021-09-04 12:18:38 +03:00
Andrew Kelley 332eafeb7f stage2: first pass at implementing usingnamespace
Ran into a design flaw here which will need to get solved by having
AstGen annotate ZIR with which instructions are closed over.
2021-09-01 17:54:06 -07:00
Jakub Konka 5533f77054 Merge remote-tracking branch 'origin/master' into zld-incremental-2 2021-07-23 17:06:19 +02:00
Andrew Kelley a5fb28070f add -femit-llvm-bc CLI option and implement it
* Added doc comments for `std.Target.ObjectFormat` enum
 * `std.Target.oFileExt` is removed because it is incorrect for Plan-9
   targets. Instead, use `std.Target.ObjectFormat.fileExt` and pass a
   CPU architecture.
 * Added `Compilation.Directory.joinZ` for when a null byte is desired.
 * Improvements to `Compilation.create` logic for computing `use_llvm`
   and reporting errors in contradictory flags. `-femit-llvm-ir` and
   `-femit-llvm-bc` will now imply `-fLLVM`.
 * Fix compilation when passing `.bc` files on the command line.
 * Improvements to the stage2 LLVM backend:
   - cleaned up error messages and error reporting. Properly bubble up
     some errors rather than dumping to stderr; others turn into panics.
   - properly call ZigLLVMCreateTargetMachine and
     ZigLLVMTargetMachineEmitToFile and implement calculation of the
     respective parameters (cpu features, code model, abi name, lto,
     tsan, etc).
   - LLVM module verification only runs in debug builds of the compiler
   - use LLVMDumpModule rather than printToString because in the case
     that we incorrectly pass a null pointer to LLVM it may crash during
     dumping the module and having it partially printed is helpful in
     this case.
   - support -femit-asm, -fno-emit-bin, -femit-llvm-ir, -femit-llvm-bc
   - Support LLVM backend when used with Mach-O and WASM linkers.
2021-07-22 19:51:32 -07:00
Jakub Konka def1359187 Merge remote-tracking branch 'origin/master' into zld-incremental-2 2021-07-22 09:34:44 +02:00
Andrew Kelley 36295d712f remove 'pe' object format
Portable Executable is an executable format, not an object format.
Everywhere in the entire zig codebase, we treated coff and pe as if they
were the same. Remove confusion by not including pe in the
std.Target.ObjectFormat enum.
2021-07-21 12:45:32 -07:00
Andrew Kelley 0f38f68696 stage2: Air and Liveness are passed ephemerally
to the link infrastructure, instead of being stored with Module.Fn. This
moves towards a strategy to make more efficient use of memory by not
storing Air or Liveness data in the Fn struct, but computing it on
demand, immediately sending it to the backend, and then immediately
freeing it.

Backends which want to defer codegen until flush() such as SPIR-V
must move the Air/Liveness data upon `updateFunc` being called and keep
track of that data in the backend implementation itself.
2021-07-20 12:19:16 -07:00
Jakub Konka e3575cdad4 zld: decommision use_lld for MachO
Invoke `linkAsArchive` directly in MachO backend when LLVM is available
and we are asked to create a static lib.
2021-07-15 18:49:47 +02:00
Andrew Kelley 28dd9d478d C backend: TypedefMap is now ArrayHashMap
The C backend depends on insertion order into this map so that type
definitions will be declared before they are used.
2021-07-12 12:40:32 -07:00
Andrew Kelley 476faef97a plan9 cleanups
* rename files to adhere to conventions
 * remove unnecessary function / optionality
 * fix merge conflict
 * better panic message
 * remove unnecessary TODO comment
 * proper namespacing of declarations
 * clean up documentation comments
 * no copyright header needed for a brand new zig file that is not
   copied from anywhere
2021-07-08 14:24:16 -07:00
Jacob G-W db2d5b49c6 plan9 linker: use a global offset table
this simplifies stuff and makes us not have to use relocations
2021-07-08 14:12:07 -07:00
Jacob G-W 798162e509 plan9 linker: make runnable binaries
We can now run binaries! (they segfault, but still run!)
2021-07-08 14:10:49 -07:00
jacob gw 34c21affa2 initial plan9 boilerplate
The code now compiles and fails with Plan9ObjectFormatUnimplemented
2021-07-08 14:10:49 -07:00
Andrew Kelley 8ce880ca75 avoid calling into stage1 backend when AstGen fails
The motivation for this commit is that there exists source files which
produce ast-check errors, but crash stage1 or otherwise trigger stage1
bugs. Previously to this commit, Zig would run AstGen, collect the
compile errors, run stage1, report stage1 compile errors and exit if
any, and then report AstGen compile errors.

The main change in this commit is to report AstGen errors prior to
invoking stage1, and in fact if any AstGen errors occur, do not invoke
stage1 at all.

This caused most of the compile error tests to fail due to things such
as unused local variables and mismatched stage1/stage2 error messages.
It was taking a long time to update the test cases one-by-one, so I
took this opportunity to unify the stage1 and stage2 testing harness,
specifically with regards to compile errors. In this way we can start
keeping track of which tests pass for 1, 2, or both.
`zig build test-compile-errors` no longer works; it is now integrated
into `zig build test-stage2`.

This is one step closer to executing compile error tests in parallel; in
fact the ThreadPool object is already in scope.

There are some cases where the stage1 compile errors were actually
better; those are left failing in this commit, to be addressed in a
follow-up commit.

Other changes in this commit:

 * build.zig: improve support for -Dstage1 used with the test step.
 * AstGen: minor cosmetic changes to error messages.
 * stage2: add -fstage1 and -fno-stage1 flags. This now allows one to
   download a binary of the zig compiler and use the llvm backend of
   self-hosted. This was also needed for hooking up the test harness.
   However, I realized that stage1 calls exit() and also has memory
   leaks, so had to complicate the test harness by not using this flag
   after all and instead invoking as a child process.
   - These CLI flags will disappear once we start shipping the
     self-hosted compiler as the main compiler. Until then, they can be
     used to try out the work-in-progress stage2.
 * stage2: select the LLVM backend by default for release modes, as long
   as the target architecture is supported by LLVM.
 * test harness: support setting the optimize mode
2021-07-02 13:27:28 -07:00
Takeshi Yoneda bc7761d8e0 Add support for WASI reactor in pure Zig-exe. (#9178)
* Add command line help for "-mexec-model"
* Define WasmExecModel enum in std.builtin.
* Drop the support for the old crt1.o in favor of crt1-command.o

Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2021-06-30 20:02:48 -04:00
Jakub Konka 75a13a456b zld: remove system linker hack and lld hooks 2021-06-29 07:59:46 +02:00
Isaac Freund 260c5aed86 stage2: add --sysroot link option
This feature is necessary for cross-compiling code that dynamically
links system libraries, at least with the current feature set of lld.
2021-06-25 19:13:43 +02:00
Jacob G-W 9fffffb07b fix code broken from previous commit 2021-06-21 17:03:03 -07:00
Jakub Konka 9f8de83d93 cc,wasi: use wasi_libc.CRTFile directly instead of WasiExecModel 2021-06-09 11:11:55 +02:00
Takeshi Yoneda bf568ec62a cc,wasi: support WASI reactors via -mexec-model flag.
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2021-06-09 17:07:06 +09:00
Jakub Konka 2ee1f7898b cc,wasi: store CRTFile enum in wasi_emulated_libs
* then, in `link/Wasm.zig` map `CRTFile` to full emulated libs name
* move logic for removing any mention of WASI snapshot
  `wasi_snapshot_preview1` from `Compilation.zig` into `link/Wasm.zig`
2021-06-09 01:25:38 +02:00
Jakub Konka 4e4722a65e cc,wasi: build referenced-only emulated components
Move parsing of system libs into `main.zig` next to where we decide
if we should link libC, and, if targeting WASI, if the specified
libname equals one of the emulated components, save it on the side
and remove it from the system libs. Then, build *only* those parts
of WASI libc that were preserved in the previous step.

This also fixes building of different crt1 bits needed to support
reactors and commands.
2021-06-09 01:25:38 +02:00
Martin Wickham fc9430f567 Breaking hash map changes for 0.8.0
- hash/eql functions moved into a Context object
- *Context functions pass an explicit context
- *Adapted functions pass specialized keys and contexts
- new getPtr() function returns a pointer to value
- remove functions renamed to fetchRemove
- new remove functions return bool
- removeAssertDiscard deleted, use assert(remove(...)) instead
- Keys and values are stored in separate arrays
- Entry is now {*K, *V}, the new KV is {K, V}
- BufSet/BufMap functions renamed to match other set/map types
- fixed iterating-while-modifying bug in src/link/C.zig
2021-06-03 17:02:16 -05:00
Jakub Konka 6d5002028a cc,wasi: link compiled WASI libc with wasm-ld 2021-05-20 16:54:00 +02:00
Andrew Kelley d839835273 stage2: add some debug logs for when link functions are called 2021-05-17 11:33:21 -07:00
Andrew Kelley 597082adf4 Merge remote-tracking branch 'origin/master' into stage2-whole-file-astgen
Conflicts:
 * build.zig
 * src/Compilation.zig
 * src/codegen/spirv/spec.zig
 * src/link/SpirV.zig
 * test/stage2/darwin.zig
   - this one might be problematic; start.zig looks for `main` in the
     root source file, not `_main`. Not sure why there is an underscore
     there in master branch.
2021-05-15 21:44:38 -07:00
Andrew Kelley a854795f52 link: fix memory leak of system_libs 2021-05-13 20:08:17 -07:00
Andrew Kelley 826179bff4 stage2: -lunwind is handled specially
* `-lc++` now implies `-lc`.
 * `-lunwind` is now pulled out into a separate `link_libunwind` flag in
   the frontend driver code. This allows a project to request zig to
   provide libunwind even if the default situation that causes it to be
   implicitly added, is not active.
 * build.zig: ask for -lunwind when building the self-hosted compiler on
   Linux. Otherwise we get linker errors with unresolved symbols to
   libunwind.
2021-05-13 22:42:57 -04:00
Andrew Kelley f86469bc5e stage2: semaDecl properly analyzes the decl block
Also flattened out Decl TypedValue fields into
ty, val, has_tv
and add relevant fields to Decl for alignment and link section.
2021-04-28 16:57:01 -07:00
Andrew Kelley bcfebb4b2b stage2: improvements aimed at std lib integration
* AstGen: emit decl lookup ZIR instructions rather than directly
   looking up decls in AstGen. This is necessary because we want to
   reuse the same immutable ZIR code for multiple generic instantiations
   (and comptime function calls).
 * AstGen: fix using members_len instead of fields_len for struct decls.
 * structs: the struct_decl ZIR instruction is now also a block. This is
   so that the type expressions, default field value expressions, and
   alignment expressions can be evaluated in a scope that contains the
   decls from the struct namespace itself.
 * Add "std" and "builtin" packages to the builtin package.
 * Don't try to build glibc, musl, or mingw-w64 when using `-ofmt=c`.
 * builtin.zig is generated without `usingnamespace`.
 * builtin.zig takes advantage of `std.zig.fmtId` for CPU features.
 * A first pass at implementing `usingnamespace`. It's problematic and
   should either be deleted, or polished, before merging this branch.
 * Sema: allow explicitly specifying the namespace in which to look up
   Decls. This is used by `struct_decl` in order to put the decls from
   the struct namespace itself in scope when evaluating the type
   expressions, default value expressions, and alignment expressions.
 * Module: fix `analyzeNamespace` assuming that it is the top-level root
   declaration node.
 * Sema: implement comptime and runtime cmp operator.
 * Sema: implement peer type resolution for enums and enum literals.
 * Pull in the changes from master branch:
   262e09c482.
 * ZIR: complete out simple_ptr_type debug printing
2021-04-15 19:06:39 -07:00
Luuk de Gram ff5774d93d Refactor link/wasm.zig to use offset table
This refactor inserts an offset table into wasm's data section
where each offset points to the actual data region.
This means we can keep offset indexes consistant and do not
have to perform any computer to determine where in the data section
something like a static string exists. Instead during runtime
it will load the data offset onto the stack.
2021-04-08 22:47:08 +02:00
Luuk de Gram 1bd5552fc1 Calculate data length to ensure correct pointer offsets 2021-04-08 22:47:08 +02:00
Timon Kruiper 6aa1ea9c59 stage2: fixup some formatting errors ({x} -> {s})
These were missed in cd7c870bd8
2021-03-02 19:02:55 -07:00
Andrew Kelley 914540ddb5 astgen: fix remaining compile errors
Now it builds and what remains in this branch is:
 * fix the stage2 compiler regressions from this branch
 * finish the rest of zig fmt test cases, get them passing
 * Merge in Vexu's translate-c AST branch & fix translate-c regressions
2021-02-19 00:04:31 -07:00
Andrew Kelley 102d954220 Merge pull request #7827 from Snektron/spirv-setup
Stage 2: SPIR-V setup
2021-02-01 12:49:51 -08:00
Andrew Kelley 0cfa39304b zig cc: recognize more coff linker options
Related: #7874
2021-01-24 14:30:28 -07:00
Andrew Kelley 0d4b6ac741 add LTO support
The CLI gains -flto and -fno-lto options to override the default.
However, the cool thing about this is that the defaults are great! In
general when you use build-exe in release mode, Zig will enable LTO if
it would work and it would help.

zig cc supports detecting and honoring the -flto and -fno-lto flags as
well. The linkWithLld functions are improved to all be the same with
regards to copying the artifact instead of trying to pass single objects
through LLD with -r. There is possibly a future improvement here as
well; see the respective TODOs.

stage1 is updated to support outputting LLVM bitcode instead of machine
code when lto is enabled. This allows LLVM to optimize across the Zig and
C/C++ code boundary.

closes #2845
2021-01-23 18:18:07 -07:00
Robin Voetter b2b87b5900 SPIR-V: Linking and codegen setup 2021-01-19 15:28:17 +01:00
Robin Voetter ab607d455e SPIR-V: Initial architecture definitions and setup 2021-01-19 15:28:17 +01:00