Hardware & Compiler

[00:00:00]
>> I can tell you about where the TensorFlow is standing, what is NumPy and how they all related. So I will dig even deeper and try to connect hardware with the compilers, interpreter languages, NumPy, optimized libraries, TensorFlow, Keras, and all of that. So I will build the hierarchy of the whole programming stack for you.

[00:00:28]
So at the bottom we have the hardware, right? And hardware, we're talking about CPUs, GPUs. So something, remember, in our diagram which was the compute block, that's our hardware, right? It is something which is just basically do the number crunching and do your particular command execution. So we need to provide those commands to our hardware.

[00:00:56]
But it usually takes everything in binary form, right, or machine language. Yeah, I can use this. [LAUGH] Well, ML, Machine Learning and Machine Language that's gonna be confusing. Okay, let's not call it ML, [LAUGH] let's just say it's just binary code. How the binary code is created? Well, it's created by the compilers.

[00:01:23]
And compilers also can be split into sub parts, so we're creating the object files first, intermediate representation, right? And then those object files dynamically linked or statically linked and that's what produce your executable, right? So compilers, I'm gonna simplify this quite a bit. So to our compilers we providing source code.

[00:01:52]
Let's say for now, we're gonna be considering two type of source codes and because of that, we'll have different pathways. So let's say we have C and C++ code. So there is compiling and interpreting languages, right? So Python using an interpreter, which actually is executing for us C ++ and C using compilers to create binary code.

[00:02:27]
So executable. With the compiler, if you provide the C source code, it will create the object file and then those object files will be linked. So linker will be used to actually create the executable. But all of that is usually hidden under the compiler, right? So probably I should use different color.

[00:03:00]
Let's go this way, so that's what compiler is doing, right? Just taking C code, creating binary object file, and then linking. Because you might have several source code files and object files will be created for each of them. And linker actually need to figure out if we're referencing different functions from different files, right?

[00:03:22]
And basically make all of those connections to create one executable. But quite often to save space, if we're using common libraries, some of the libraries will not be compiled from the source code but will be grabbed from the operating system. So in OS, for instance, if we're working on Windows, it will be a DLL, right, Dynamically Linked Libraries.

[00:03:48]
In Linux, it will be .o or .a files, and linker when it's actually looking at the function calls. And if those functions are, for instance, system calls, it will just grab those from the operating system and link them together. Or figure out the way to load them from the operating system to save the size of your executable, depending if you want dynamic or static loading.

[00:04:17]
Static loading means that we just gonna grab those and build them into our executable. And dynamic loading means that we hope that this library will be in our operating system or actually final user who are running the executable will have them on their operating system and it will be grabbed from there.

[00:04:37]
So that's kind of the compilation process. And with the Python, you just have interpreter which running those commands, just interpreting how those high level commands should be executed on the low level.