123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208 |
- x86 Topology
- ============
- This documents and clarifies the main aspects of x86 topology modelling and
- representation in the kernel. Update/change when doing changes to the
- respective code.
- The architecture-agnostic topology definitions are in
- Documentation/cputopology.txt. This file holds x86-specific
- differences/specialities which must not necessarily apply to the generic
- definitions. Thus, the way to read up on Linux topology on x86 is to start
- with the generic one and look at this one in parallel for the x86 specifics.
- Needless to say, code should use the generic functions - this file is *only*
- here to *document* the inner workings of x86 topology.
- Started by Thomas Gleixner <tglx@linutronix.de> and Borislav Petkov <bp@alien8.de>.
- The main aim of the topology facilities is to present adequate interfaces to
- code which needs to know/query/use the structure of the running system wrt
- threads, cores, packages, etc.
- The kernel does not care about the concept of physical sockets because a
- socket has no relevance to software. It's an electromechanical component. In
- the past a socket always contained a single package (see below), but with the
- advent of Multi Chip Modules (MCM) a socket can hold more than one package. So
- there might be still references to sockets in the code, but they are of
- historical nature and should be cleaned up.
- The topology of a system is described in the units of:
- - packages
- - cores
- - threads
- * Package:
- Packages contain a number of cores plus shared resources, e.g. DRAM
- controller, shared caches etc.
- AMD nomenclature for package is 'Node'.
- Package-related topology information in the kernel:
- - cpuinfo_x86.x86_max_cores:
- The number of cores in a package. This information is retrieved via CPUID.
- - cpuinfo_x86.phys_proc_id:
- The physical ID of the package. This information is retrieved via CPUID
- and deduced from the APIC IDs of the cores in the package.
- - cpuinfo_x86.logical_id:
- The logical ID of the package. As we do not trust BIOSes to enumerate the
- packages in a consistent way, we introduced the concept of logical package
- ID so we can sanely calculate the number of maximum possible packages in
- the system and have the packages enumerated linearly.
- - topology_max_packages():
- The maximum possible number of packages in the system. Helpful for per
- package facilities to preallocate per package information.
- * Cores:
- A core consists of 1 or more threads. It does not matter whether the threads
- are SMT- or CMT-type threads.
- AMDs nomenclature for a CMT core is "Compute Unit". The kernel always uses
- "core".
- Core-related topology information in the kernel:
- - smp_num_siblings:
- The number of threads in a core. The number of threads in a package can be
- calculated by:
- threads_per_package = cpuinfo_x86.x86_max_cores * smp_num_siblings
- * Threads:
- A thread is a single scheduling unit. It's the equivalent to a logical Linux
- CPU.
- AMDs nomenclature for CMT threads is "Compute Unit Core". The kernel always
- uses "thread".
- Thread-related topology information in the kernel:
- - topology_core_cpumask():
- The cpumask contains all online threads in the package to which a thread
- belongs.
- The number of online threads is also printed in /proc/cpuinfo "siblings."
- - topology_sibling_mask():
- The cpumask contains all online threads in the core to which a thread
- belongs.
- - topology_logical_package_id():
- The logical package ID to which a thread belongs.
- - topology_physical_package_id():
- The physical package ID to which a thread belongs.
- - topology_core_id();
- The ID of the core to which a thread belongs. It is also printed in /proc/cpuinfo
- "core_id."
- System topology examples
- Note:
- The alternative Linux CPU enumeration depends on how the BIOS enumerates the
- threads. Many BIOSes enumerate all threads 0 first and then all threads 1.
- That has the "advantage" that the logical Linux CPU numbers of threads 0 stay
- the same whether threads are enabled or not. That's merely an implementation
- detail and has no practical impact.
- 1) Single Package, Single Core
- [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
- 2) Single Package, Dual Core
- a) One thread per core
- [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
- -> [core 1] -> [thread 0] -> Linux CPU 1
- b) Two threads per core
- [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
- -> [thread 1] -> Linux CPU 1
- -> [core 1] -> [thread 0] -> Linux CPU 2
- -> [thread 1] -> Linux CPU 3
- Alternative enumeration:
- [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
- -> [thread 1] -> Linux CPU 2
- -> [core 1] -> [thread 0] -> Linux CPU 1
- -> [thread 1] -> Linux CPU 3
- AMD nomenclature for CMT systems:
- [node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0
- -> [Compute Unit Core 1] -> Linux CPU 1
- -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2
- -> [Compute Unit Core 1] -> Linux CPU 3
- 4) Dual Package, Dual Core
- a) One thread per core
- [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
- -> [core 1] -> [thread 0] -> Linux CPU 1
- [package 1] -> [core 0] -> [thread 0] -> Linux CPU 2
- -> [core 1] -> [thread 0] -> Linux CPU 3
- b) Two threads per core
- [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
- -> [thread 1] -> Linux CPU 1
- -> [core 1] -> [thread 0] -> Linux CPU 2
- -> [thread 1] -> Linux CPU 3
- [package 1] -> [core 0] -> [thread 0] -> Linux CPU 4
- -> [thread 1] -> Linux CPU 5
- -> [core 1] -> [thread 0] -> Linux CPU 6
- -> [thread 1] -> Linux CPU 7
- Alternative enumeration:
- [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
- -> [thread 1] -> Linux CPU 4
- -> [core 1] -> [thread 0] -> Linux CPU 1
- -> [thread 1] -> Linux CPU 5
- [package 1] -> [core 0] -> [thread 0] -> Linux CPU 2
- -> [thread 1] -> Linux CPU 6
- -> [core 1] -> [thread 0] -> Linux CPU 3
- -> [thread 1] -> Linux CPU 7
- AMD nomenclature for CMT systems:
- [node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0
- -> [Compute Unit Core 1] -> Linux CPU 1
- -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2
- -> [Compute Unit Core 1] -> Linux CPU 3
- [node 1] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 4
- -> [Compute Unit Core 1] -> Linux CPU 5
- -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 6
- -> [Compute Unit Core 1] -> Linux CPU 7
|