[api] What is an application binary interface (ABI)?



Answers

If you know assembly and how things work at the OS-level, you are conforming to a certain ABI. The ABI govern things like how parameters are passed, where return values are placed. For many platforms there is only one ABI to choose from, and in those cases the ABI is just "how things work".

However, the ABI also govern things like how classes/objects are laid out in C++. This is necessary if you want to be able to pass object references across module boundaries or if you want to mix code compiled with different compilers.

Also, if you have an 64-bit OS which can execute 32-bit binaries, you will have different ABIs for 32- and 64-bit code.

In general, any code you link into the same executable must conform to the same ABI. If you want to communicate between code using different ABIs, you must use some form of RPC or serialization protocols.

I think you are trying too hard to squeeze in different types of interfaces into a fixed set of characteristics. For example, an interface doesn't necessarily have to be split into consumers and producers. An interface is just a convention by which two entities interact.

ABIs can be (partially) ISA-agnostic. Some aspects (such as calling conventions) depend on the ISA, while other aspects (such as C++ class layout) do not.

A well defined ABI is very important for people writing compilers. Without a well defined ABI, it would be impossible to generate interoperable code.

EDIT: Some notes to clarify:

  • "Binary" in ABI does not exclude the use of strings or text. If you want to link a DLL exporting a C++ class, somewhere in it the methods and type signatures must be encoded. That's where C++ name-mangling comes in.
  • The reason why you never provided an ABI is that the vast majority of programmers will never do it. ABIs are provided by the same people designing the platform (i.e. operating system), and very few programmers will ever have the privilege to design a widely-used ABI.
Question

I never clearly understood what an ABI is. Please don't point me to a Wikipedia article. If I could understand it, I wouldn't be here posting such a lengthy post.

This is my mindset about different interfaces:

A TV remote is an interface between the user and the TV. It is an existing entity, but useless (doesn't provide any functionality) by itself. All the functionality for each of those buttons on the remote is implemented in the television set.

Interface: It is an "existing entity" layer between the functionality and consumer of that functionality. An interface by itself is doesn't do anything. It just invokes the functionality lying behind.

Now depending on who the user is there are different type of interfaces.

Command Line Interface (CLI) commands are the existing entities, the consumer is the user and functionality lies behind.

functionality: my software functionality which solves some purpose to which we are describing this interface.

existing entities: commands

consumer: user

Graphical User Interface(GUI) window, buttons, etc. are the existing entities, and again the consumer is the user and functionality lies behind.

functionality: my software functionality which solves some purpose to which we are describing this interface.

existing entities: window,buttons etc..

consumer: user

Application Programming Interface(API) functions or to be more correct, interfaces (in interfaced based programming) are the existing entities, consumer here is another program not a user, and again functionality lies behind this layer.

functionality: my software functionality which solves some purpose to which we are describing this interface.

existing entities: functions, Interfaces(array of functions).

consumer: another program/application.

Application Binary Interface (ABI) Here is where my problem starts.

functionality: ???

existing entities: ???

consumer: ???

  • I've written software in different languages and provided different kind of interfaces (CLI, GUI, and API), but I'm not sure, if I ever, provided any ABI.

Wikipedia says:

ABIs cover details such as

  • data type, size, and alignment;
  • the calling convention, which controls how functions' arguments are passed and return values retrieved;
  • the system call numbers and how an application should make system calls to the operating system;

Other ABIs standardize details such as

  • the C++ name mangling,
  • exception propagation, and
  • calling convention between compilers on the same platform, but do not require cross-platform compatibility.
  • Who needs these details? Please don't say the OS. I know assembly programming. I know how linking & loading works. I know what exactly happens inside.

  • Why did C++ name mangling come in? I thought we are talking at the binary level. Why do languages come in?

Anyway, I've downloaded the [PDF] System V Application Binary Interface Edition 4.1 (1997-03-18) to see what exactly it contains. Well, most of it didn't make any sense.

  • Why does it contain two chapters (4th & 5th) to describe the ELF file format? In fact, these are the only two significant chapters of that specification. The rest of the chapters are "processor specific". Anyway, I thought that it is a completely different topic. Please don't say that ELF file format specifications are the ABI. It doesn't qualify to be an interface according to the definition.

  • I know, since we are talking at such a low level it must be very specific. But I'm not sure how is it "instruction set architecture (ISA)" specific?

  • Where can I find Microsoft Windows' ABI?

So, these are the major queries that are bugging me.




In order to call code in shared libraries, or call code between compilation units, the object file needs to contain labels for the calls. C++ mangles the names of method labels in order to enforce data hiding and allow for overloaded methods. That is why you cannot mix files from different C++ compilers unless they explicitly support the same ABI.




The ABI needs to be consistent between caller and callee to be certain that the call succeeds. Stack use, register use, end-of-routine stack pop. All these are the most important parts of the ABI.




You actually don't need an ABI at all if--

  • Your program doesn't have functions, and--
  • Your program is a single executable that is running alone (i.e. an embedded system) where it's literally the only thing running and it doesn't need to talk to anything else.

An oversimplified summary:

API: "Here are all the functions you may call."

ABI: "This is how to call a function."

The ABI is set of rules that compilers and linkers adhere to in order to compile your program so that will work properly. ABIs cover multiple topics:

  • Arguably the biggest and most important part of an ABI is the procedure call standard sometimes known as the "calling convention". Calling conventions standardize how "functions" are translated to assembly code.
  • ABIs also dictate the how the names of exposed functions in libraries should be represented so that other code can call those libraries and know what arguments should be passed. This is called "name mangling".
  • ABIs also dictate what type of data types can be used, how they must be aligned, and other low-level details.

Taking a deeper look at calling convention, which I consider to be the core of an ABI:

The machine itself has no concept of "functions". When you write a function in a high-level language like c, the compiler generates a line of assembly code like _MyFunction1:. This is a label, which will eventually get resolved into an address by the assembler. This label marks the "start" of your "function" in the assembly code. In high-level code, when you "call" that function, what you're really doing is causing the CPU to jump to the address of that label and continue executing there.

In preparation for the jump, the compiler must do a bunch of important stuff. The calling convention is like a checklist that the compiler follows to do all this stuff:

  • First, the compiler inserts a little bit of assembly code to save the current address, so that when your "function" is done, the CPU can jump back to the right place and continue executing.
  • Next, the compiler generates assembly code to pass the arguments.
    • Some calling conventions dictate that arguments should be put on the stack (in a particular order of course).
    • Other conventions dictate that the arguments should be put in particular registers (depending on their data types of course).
    • Still other conventions dictate that a specific combination of stack and registers should be used.
  • Of course, if there was anything important in those registers before, those values are now overwritten and lost forever, so some calling conventions may dictate that the compiler should save some of those registers prior to putting the arguments in them.
  • Now the compiler inserts a jump instruction telling the CPU to go to that label it made previously (_MyFunction1:). At this point, you can consider the CPU to be "in" your "function".
  • At the end of the function, the compiler puts some assembly code that will make the CPU write the return value in the correct place. The calling convention will dictate whether the return value should be put into a particular register (depending on its type), or on the stack.
  • Now it's time for clean-up. The calling convention will dictate where the compiler places the cleanup assembly code.
    • Some conventions say that the caller must clean up the stack. This means that after the "function" is done and the CPU jumps back to where it was before, the very next code to be executed should be some very specific cleanup code.
    • Other conventions say that the some particular parts of the cleanup code should be at the end of the "function" before the jump back.

There are many different ABIs / calling conventions. Some main ones are:

  • For the x86 or x86-64 CPU (32-bit environment):
    • CDECL
    • STDCALL
    • FASTCALL
    • VECTORCALL
    • THISCALL
  • For the x86-64 CPU (64-bit environment):
    • SYSTEMV
    • MSNATIVE
    • VECTORCALL
  • For the ARM CPU (32-bit)
    • AAPCS
  • For the ARM CPU (64-bit)
    • AAPCS64

Here is a great page that actually shows the differences in the assembly generated when compiling for different ABIs.

Another thing to mention is that an ABI isn't only relevant inside your program's executable module. It's also used by the linker to make sure your program calls library functions correctly. You have multiple shared libraries running on your computer, and as long as your compiler knows what ABI they each use, it can call functions from them properly without blowing up the stack.

Your compiler understanding how to call library functions is extremely important. On a hosted platform (that is, one where an OS loads programs), your program can't even blink without making a kernel call.




Application binary interface (ABI)

Functionality:

  • Translation from the programmer's model to the underlying system's domain data type, size, alignment, the calling convention, which controls how functions' arguments are passed and return values retrieved; the system call numbers and how an application should make system calls to the operating system; the high-level language compilers' name mangling scheme, exception propagation, and calling convention between compilers on the same platform, but do not require cross-platform compatibility...

Existing entities:

  • Logical blocks that directly participate in program's execution: ALU, general purpose registers, registers for memory/ I/O mapping of I/O, etc...

consumer:

  • Language processors linker, assembler...

These are needed by whoever has to ensure that build tool-chains work as a whole. If you write one module in assembly language, another in Python, and instead of your own boot-loader want to use an operating system, then your "application" modules are working across "binary" boundaries and require agreement of such "interface".

C++ name mangling because object files from different high-level languages might be required to be linked in your application. Consider using GCC standard library making system calls to Windows built with Visual C++.

ELF is one possible expectation of the linker from an object file for interpretation, though JVM might have some other idea.

For a Windows RT Store app, try searching for ARM ABI if you really wish to make some build tool-chain work together.




Let me at least answer a part of your question. With an example of how the Linux ABI affects the systemcalls, and why that is usefull.

A systemcall is a way for a userspace program to ask the kernelspace for something. It works by putting the numeric code for the call and the argument in a certain register and triggering an interrupt. Than a switch occurs to kernelspace and the kernel looks up the numeric code and the argument, handles the request, puts the result back into a register and triggers a switch back to userspace. This is needed for example when the application wants to allocate memory or open a file (syscalls "brk" and "open").

Now the syscalls have short names "brk", etc. and corresponding opcodes, these are defined in a system specific header file. As long as these opcodes stay the same you can run the same compiled userland programs with different updated kernels without having to recompile. So you have an interface used by precompiled binarys, hence ABI.




Related