亚洲欧美成人久久综合中文网,久久精品国产WWW456C0M,久久久久久精品免费看SSS

High Memory In The Linux Kernel

JonsenElizee — Sat, 22 Jan 2011 02:44:00 GMT

Feature: High Memory In The Linux Kernel

on February 21, 2004 - 6:02am

high memory
Linux
memory
memory management
PAE
RAM
Linux feature article

As RAM increasingly becomes a commodity, the prices drop and computer users are able to buy more. 32-bit archictectures face certain limitations in regards to accessing these growing amounts of RAM. To better understand the problem and the various solutions, we begin with an overview of Linux memory management. Understanding how basic memory management works, we are better able to define the problem, and finally to review the various solutions.

This article was written by examining the Linux 2.6 kernel source code for the x86 architecture types.

Overview of Linux memory management

32-bit architectures can reference 4 GB of physical memory (2^32). Processors that have an MMU (Memory Management Unit) support the concept of virtual memory: page tables are set up by the kernel which map "virtual addresses" to "physical addresses"; this basically means that each process can access 4 GB of memory, thinking it's the only process running on the machine (much like multi-tasking, in which each process is made to think that it's the only process executing on a CPU).

The virtual address to physical address mappings are done by the kernel. When a new process is "fork()"ed, the kernel creates a new set of page tables for the process. The addresses referenced within a process in user-space are virtual addresses. They do not necessarily map directly to the same physical address. The virtual address is passed to the MMU (Memory Management Unit of the processor) which converts it to the proper physical address based on the tables set up by the kernel. Hence, two processes can refer to memory address 0x08329, but they would refer to two different locations in memory.

The Linux kernel splits the 4 GB virtual address space of a process in two parts: 3 GB and 1 GB. The lower 3 GB of the process virtual address space is accessible as the user-space virtual addresses and the upper 1 GB space is reserved for the kernel virtual addresses. This is true for all processes.

							      
      +----------+ 4 GB				      
      |          |     				      
      |	         |     				      
      |	         |     				      
      | Kernel   |     				      
      |          |     			       +----------+ 
      |	Virtual  |     			       |          |
      |          |     			       |          |
      | Space    |     			       | High     |
      |          |     			       |          |
      | (1 GB)   |     			       | Memory   |
      |          |     			       |          |
      |	         |     			       | (unused) |
      +----------+ 3 GB	   		       +----------+ 1 GB
      |          |     	   		       |          |
      |          |     	   		       |          |
      |	         |     	   		       |          |
      |          |     			       | Kernel   |
      |          |     			       |          |
      |          |     			       | Physical |
      |          |     			       |          |
      |User-space|     			       | Space    |
      |	       	 |     			       |      	  |
      | Virtual  |     			       |          |
      |          |     			       |          |
      | Space    |     			       |          |
      |          |     			       |          |    	
      | (3 GB)   |     			       +----------+ 0 GB
      |          |     				      	 
      |          |     			       	 Physical 
      |          |     			          Memory 
      |          |     				      	 
      |          |     				      	 
      |          |     				      	 
      |          |     				      	 
      |          |     				      	 
      +----------+ 0 GB					 
			 					 
       	Virtual	 					 
       	Memory

The kernel virtual area (3 - 4 GB address space) maps to the first 1 GB of physical RAM. The 3 GB addressable RAM available to each process is mapped to the available physical RAM.

The Problem

So, the basic problem here is, the kernel can just address 1 GB of virtual addresses, which can translate to a maximum of 1 GB of physical memory. This is because the kernel directly maps all available kernel virtual space addresses to the available physical memory.

Solutions

There are some solutions which address this problem:

2G / 2G, 1G / 3G split
HIGHMEM solution for using up to 4 GB of memory
HIGHMEM solution for using up to 64 GB of memory

1. 2G / 2G, 1G / 3G split

Instead of splitting the virtual address space the traditional way of 3G / 1G (3 GB for user-space, 1 GB for kernel space), third-party patches exist to split the virtual address space 2G / 2G or 1G / 3G. The 1G / 3G split is a bit extreme in that you can map up to 3 GB of physical memory, but user-space applications cannot grow beyond 1 GB. It could work for simple applications; but if one has more than 3 GB of physical RAM, he / she won't run simple applications on it, right?

The 2G / 2G split seems to be a balanced approach to using RAM more than 1 GB without using the HIGHMEM patches. However, server applications like databases always want as much virtual addressing space as possible; so this approach may not work in those scenarios.

There's a patch for 2.4.23 that includes a config-time option of selecting the user / kernel split values by Andrea Arcangeli. It is available at his kernel page. It's a simple patch and making it work on 2.6 should not be too difficult.

Before looking at solutions 2 & 3, let's take a look at some more Linux Memory Management issues.

Zones

In Linux, the memory available from all banks is classified into "nodes". These nodes indicate how much memory each bank has. This classification is mainly useful for NUMA architectures, but it's also used for UMA architectures, where the number of nodes is just 1.

Memory in each node is divided into "zones". The zones currently defined are ZONE_DMA, ZONE_NORMAL and ZONE_HIGHMEM.

ZONE_DMA is used by some devices for data transfer and is mapped in the lower physical memory range (up to 16 MB).

Memory in the ZONE_NORMAL region is mapped by the kernel in the upper region of the linear address space. Most operations can only take place in ZONE_NORMAL; so this is the most performance critical zone. ZONE_NORMAL goes from 16 MB to 896 MB.

To address memory from 1 GB onwards, the kernel has to map pages from high memory into ZONE_NORMAL.

Some area of memory is reserved for storing several kernel data structures that store information about the memory map and page tables. This on x86 is 128 MB. Hence, of the 1 GB physical memory the kernel can access, 128MB is reserved. This means that the kernel virtual address in this 128 MB is not mapped to physical memory. This leaves a maximum of 896 MB for ZONE_NORMAL. So, even if one has 1 GB of physical RAM, just 896 MB will be actually available.

Back to the solutions:

2. HIGHMEM solution for using up to 4 GB of memory

Since Linux can't access memory which hasn't been directly mapped into its address space, to use memory > 1 GB, the physical pages have to be mapped in the kernel virtual address space first. This means that the pages in ZONE_HIGHMEM have to be mapped in ZONE_NORMAL before they can be accessed.

The reserved space which we talked about earlier (in case of x86, 128 MB) has an area in which pages from high memory are mapped into the kernel address space.

To create a permanent mapping, the "kmap" function is used. Since this function may sleep, it may not be used in interrupt context. Since the number of permanent mappings is limited (if not, we could've directly mapped all the high memory in the address space), pages mapped this way should be "kunmap"ped when no longer needed.

Temporary mappings can be created via "kmap_atomic". This function doesn't block, so it can be used in interrupt context. "kunmap_atomic" un-maps the mapped high memory page. A temporary mapping is only available as long as the next temporary mapping. However, since the mapping and un-mapping functions also disable / enable preemption, it's a bug to not kunmap_atomic a page mapped via kmap_atomic.

3. HIGHMEM solution for using 64 GB of memory

This is enabled via the PAE (Physical Address Extension) extension of the PentiumPro processors. PAE addresses the 4 GB physical memory limitation and is seen as Intel's answer to AMD 64-bit and AMD x86-64. PAE allows processors to access physical memory up to 64 GB (36 bits of address bus). However, since the virtual address space is just 32 bits wide, each process can't grow beyond 4 GB. The mechanism used to access memory from 4 GB to 64 GB is essentially the same as that of accessing the 1 GB - 4 GB RAM via the HIGHMEM solution discussed above.

Should I enable CONFIG_HIGHMEM for my 1 GB RAM system?

It is advised to not enable CONFIG_HIGHMEM in the kernel to utilize the extra 128 MB you get for your 1 GB RAM system. I/O Devices cannot directly address high memory from PCI space, so bounce buffers have to be used. Plus the virtual memory management and paging costs come with extra mappings. For details on bounce buffers, refer to Mel Gorman's documentation (link below).

For more information, see

Andrea Arcangeli's article on the original HIGHMEM patches in Linux 2.3
Mel Gorman's VM Documentation for the 2.4 Linux Memory Management subsystem
Linux kernel sources

JonsenElizee 2011-01-22 10:44 发表评论

Linux lib

JonsenElizee — Thu, 30 Sep 2010 18:10:00 GMT

How does linux lib work?

�?Linux �q�行动态加�?/a>

Linux �q�不会自动�ؓ�l�定�E�序加蝲和链接库�Q�而是与应用程序本�w�共享该控制权。这个过�E�就�U�Cؓ动态加�?/em>。��用动态加载，应用�E�序能够先指定要加蝲的库�Q�然后将该库作�ؓ一个可执行文�g来��用（卌��用其中的函数�Q�。但是正如您在前面所了解到的�Q�用于动态加载的�׃�n库与标准�׃�n库（ELF �׃�n对象�Q�无异。事实上�Q?code>ld-linux 动态链接器作�ؓ ELF 加蝲器和解释器，仍然会参与到�q�个�q�程中�?/p>

动态加载（Dynamic Loading�Q�DL�Q�API ��是��Z��动态加载而存在的�Q�它允许�׃�n库对用户�I�间�E�序可用。尽��非常小�Q�但是这�?API 提供了所有需要的东西�Q�而且很多困难的工作是在后台完成的。表 1 展示了这个完整的 API�?/p>
�?1. Dl API

函数	描述
dlopen	使对象文件可被程序访�?/td>
dlsym	获取执行�?`dlopen` 函数的对象文件中的符��L��地址
dlerror	�q�回上一�ơ出现错误的字符串错�?/td>
dlclose	关闭目标文�g

该过�E�首先是调用 dlopen�Q�提供要讉K��的文件对象和模式。调�?dlopen 的结果是�E�候要使用的对象的句柄�?code>mode 参数通知动态链接器何时执行再定位。有两个可能的倹{��第一个是 RTLD_NOW�Q�它表明动态链接器��会在调�?dlopen 时完成所有必要的再定位。第二个可选的模式�?RTLD_LAZY�Q�它只在需要时执行再定位。这是通过在内部��用动态链接器重定向所有尚未再定位的请求来完成的。这��P��动态链接器��p��够在��h��时知晓何时发生了新的引用�Q�而且再定位可以正常进行。后面的调用无需重复再定位过�E��?/p>
�q�可以选择另外两种模式�Q�它们可以按�?OR �?mode 参数中�?code>RTLD_LOCAL 表明其他��M��对象都无法��加蝲的共享对象的�W�号用于再定位过�E�。如果这正是您想要的的话�Q�例如，��Z��让共享的对象能够调用原始�q�程映像中的�W�号�Q�，那就使用 RTLD_GLOBAL 吧�?/p>
dlopen 函数�q�会自动解析�׃�n库中的依赖项。这��P��如果您打开了一个依赖于其他�׃�n库的对象�Q�它��׃��自动加蝲它们。函数返回一个句柄，该句柄用于后�l�的 API 调用�?code>dlopen 的原型�ؓ�Q?/p>

#include

void *dlopen( const char *file, int mode );

有了 ELF 对象的句柄，��可以通过调用 dlsym 来识别这个对象内的符��L��地址了。该函数采用一个符号名�U�ͼ�如对象内的一个函数的名称。返回��gؓ对象�W�号的解析地址�Q?/p>

void *dlsym( void *restrict handle, const char *restrict name );

如果调用�?API 时发生了错误�Q�可以��?dlerror 函数�q�回一个表�C�此错误的�h�c�d��ȝ��字符丌Ӏ�该函数没有参数�Q�它会在发生前面的错误时�q�回一个字�W�串�Q�在没有错误发生时返�?NULL�Q?/p>

char *dlerror();

最后，如果无需再调用共享对象的话，应用�E�序可以调用 dlclose 来通知操作�pȝ��不再需要句柄和对象引用了。它完全是按引用来计数的�Q�所以同一个共享对象的多个用户�怺�间不会发生冲�H�（只要�q�有一个用户在使用它，它就会待在内存中�Q�。�Q何通过已关闭的对象�?dlsym 解析的符号都��不再可用�?/p>

char *dlclose( void *handle );

回页�?/strong>

动态加载示�?/a>

�? 解了 API 之后�Q�下面让我们来看一�?DL API 的例子。在�q�个应用�E�序中，您主要实��C��一�? shell�Q�它允许操作员来指定库、函数和参数。换句话��_��也就是用戯��够指定一个库�q�调用该库（先前未链接于该应用程序的�Q�内的�Q意一个函数。首先��? DL API 来解析该库中的函敎ͼ�然后使用用户定义的参敎ͼ�用来发送结果）来调用它。清�?2 展示了完整的应用�E�序�?/p>
清单 2. 使用 DL API �?Shell

#include
#include
#include

#define MAX_STRING 80

void invoke_method( char *lib, char *method, float argument )
{
void *dl_handle;
float (*func)(float);
char *error;

/* Open the shared object */
dl_handle = dlopen( lib, RTLD_LAZY );
if (!dl_handle) {
printf( "!!! %s\n", dlerror() );
return;
}

/* Resolve the symbol (method) from the object */
func = dlsym( dl_handle, method );
error = dlerror();
if (error != NULL) {
printf( "!!! %s\n", error );
return;
}

/* Call the resolved method and print the result */
printf(" %f\n", (*func)(argument) );

/* Close the object */
dlclose( dl_handle );

return;
}

int main( int argc, char *argv[] )
{
char line[MAX_STRING+1];
char lib[MAX_STRING+1];
char method[MAX_STRING+1];
float argument;

while (1) {

printf("> ");

line[0]=0;
fgets( line, MAX_STRING, stdin);

if (!strncmp(line, "bye", 3)) break;

sscanf( line, "%s %s %f", lib, method, &argument);

invoke_method( lib, method, argument );

}

}

要构��个应用程序，需要通过 GNU Compiler Collection�Q�GCC�Q��用如下的�~�译行。选项 -rdynamic 用来通知链接器将所有符��h��加到动态符可��中（目的是能够通过使用 dlopen 来实现向后跟�t�）�?code>-ldl 表明一定要��?dllib 链接于该�E�序�?/p>

gcc -rdynamic -o dl dl.c -ldl

再回�?清单 2�Q?code>main 函数仅充当解释器�Q�解析来自输入行的三个参敎ͼ�库名、函数名和��Q点参敎ͼ�。如果出�?bye 的话�Q�应用程序就会退出。否则的话，�q�三个参数就会传递给使用 DL API �?invoke_method 函数�?/p>
首先调用 dlopen 来访问目标文件。如果返�?NULL 句柄�Q�表�C�无法找到对象，�q�程�l�束。否则的话，��会得到对象的一个句柄，可以�q�一步询问对象。然后��?dlsym API 函数�Q�尝试解析新打开的对象文件中的符受��您��会得到一个有效的指向该符��L��指针�Q�或者是得到一�?NULL �q�返回一个错误�?/p>
�? ELF 对象中解析了�W�号后，下一步就只需要调用函数。要注意一下这个代码和前面讨论的动态链接的差别。在�q�个例子中，您强行将目标文�g中的�W�号地址用作函数�? 针，然后调用它。而在前面的例子是��对象名作�ؓ函数�Q�由动态链接器来确保符��h��向正��的位置。虽然动态链接器能够为您做所有麻烦的工作�Q�但�q�个�Ҏ��会让�? 构徏出极其动态的应用�E�序�Q�它们可以再�q�行时被扩展�?/p>
调用 ELF 对象中的目标函数后，通过调用 dlclose 来关闭对它的讉K��?/p>
�? �?3 展示了一个如何��用这个测试程序的例子。在�q�个例子中，首先�~�译�E�序而后执行它。接着调用�?math 库（libm.so�Q�中的几个函数。完成演�C�后�Q�程序现在能够用动态加载来调用�׃�n对象�Q�库�Q�中的�Q意函��C��。这是一个很强大的功能，通过它还能够�l�程�? 扩充新的功能�?/p>
清单 3. 使用��单的�E�序来调用库函数

mtj@camus:~/dl$ gcc -rdynamic -o dl dl.c -ldl
mtj@camus:~/dl$ ./dl
> libm.so cosf 0.0
1.000000
> libm.so sinf 0.0
0.000000
> libm.so tanf 1.0
1.557408
> bye
mtj@camus:~/dl$

回页�?/strong>

工具

Linux 提供了很多种查看和解�?ELF 对象�Q�包括共享库�Q�的工具。其中最有用的一个当�?ldd 命��o�Q�您可以使用它来发送共享库依赖��V��例如，�?dl 应用�E�序上��?ldd 命��o会显�C�如下内容：

mtj@camus:~/dl$ ldd dl
linux-gate.so.1 => (0xffffe000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7fdb000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7eac000)
/lib/ld-linux.so.2 (0xb7fe7000)
mtj@camus:~/dl$

ldd 所告诉您的是：�?ELF 映像依赖�?linux-gate.so�Q�一个特�D�的�׃�n对象�Q�它处理�pȝ��调用�Q�它在文件系�l�中无关联文�Ӟ��、libdl.so�Q�DL API�Q�、GNU C 库（libc.so�Q�以�?Linux 动态加载器�Q�因为它里面有共享库依赖��）�?/p>
readelf 命��o是一个有很多�Ҏ��的实用�E�序�Q�它让您能够解析和读�?ELF 对象�?code>readelf 有一个有��的用途，��是用来识别对象内可再定位的��V��对于我们这个简单的�E�序来说�Q?a >清单 2 展示的程序）�Q�您可以看到需要再定位的符号�ؓ�Q?/p>

mtj@camus:~/dl$ readelf -r dl

Relocation section '.rel.dyn' at offset 0x520 contains 2 entries:
Offset Info Type Sym.Value Sym. Name
08049a3c 00001806 R_386_GLOB_DAT 00000000 __gmon_start__
08049a78 00001405 R_386_COPY 08049a78 stdin

Relocation section '.rel.plt' at offset 0x530 contains 8 entries:
Offset Info Type Sym.Value Sym. Name
08049a4c 00000207 R_386_JUMP_SLOT 00000000 dlsym
08049a50 00000607 R_386_JUMP_SLOT 00000000 fgets
08049a54 00000b07 R_386_JUMP_SLOT 00000000 dlerror
08049a58 00000c07 R_386_JUMP_SLOT 00000000 __libc_start_main
08049a5c 00000e07 R_386_JUMP_SLOT 00000000 printf
08049a60 00001007 R_386_JUMP_SLOT 00000000 dlclose
08049a64 00001107 R_386_JUMP_SLOT 00000000 sscanf
08049a68 00001907 R_386_JUMP_SLOT 00000000 dlopen
mtj@camus:~/dl$

从这个列表中�Q�您可以看到各种各样的需要再定位�Q�到 libc.so�Q�的 C 库调用，包括�?DL API�Q�libdl.so�Q�的调用。函�? __libc_start_main 是一�? C 库函敎ͼ�它优先于�E�序�? main 函数�Q�一个提供必要初始化�?shell�Q�而被调用�?/p>
其他操作对象文�g的实用程序包括：objdump�Q�它展示了关于对象文件的信息�Q?code>nm�Q�它列出来自对象文�g�Q�包括调试信息）的符受��还可以��?EFL �E�序作�ؓ参数�Q�直接调�?Linux 动态链接器�Q�从而手动启动映像：

mtj@camus:~/dl$ /lib/ld-linux.so.2 ./dl
> libm.so expf 0.0
1.000000
>

另外�Q�可以��?ld-linux.so �?--list 选项来罗�?ELF 映像的依赖项�Q?code>ldd 命��o也如此）。切讎ͼ�它仅仅是一个用��L��间程序，是由内核在需要时引导的�?/p>

JonsenElizee 2010-10-01 02:10 发表评论