larbin源代碼分析<三> url類分析
一 分析utils包中的url類
該類代表實(shí)際中的一個(gè)url，成員變量主要有 ，char * file ，char * host , uint16_t port , int8_t depth, char * cookie
還有一個(gè)public 屬性的 in_addr 表示一個(gè)ipv4的地址。
成員函數(shù)中主要有一些，比如構(gòu)造函數(shù)，返回url，添加cookie等操作。
二 實(shí)例代碼如下
// Larbin
// Sebastien Ailleret
// 15-11-99 -> 14-03-02
/**//* This class describes an URL */
#ifndef URL_H
#define URL_H
#include <netinet/in.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <stdlib.h>
#include "types.h"
bool fileNormalize (char *file);
class url 
{
private:
char *host;
char *file;
uint16_t port; // the order of variables is important for physical size
int8_t depth;
/**//* parse the url */
void parse (char *s);
/**//** parse a file with base */
void parseWithBase (char *u, url *base);
/**//* normalize file name */
bool normalize (char *file);
/**//* Does this url starts with a protocol name */
bool isProtocol (char *s);
/**//* constructor used by giveBase */
url (char *host, uint port, char *file);
public:
/**//* Constructor : Parses an url (u is deleted) */
url (char *u, int8_t depth, url *base);
/**//* constructor used by input */
url (char *line, int8_t depth);
/**//* Constructor : read the url from a file (cf serialize) */
url (char *line);
/**//* Destructor */
~url ();
/**//* inet addr (once calculated) */
struct in_addr addr;
/**//* Is it a valid url ? */
bool isValid ();
/**//* print an URL */
void print ();
/**//* return the host */
inline char *getHost () { return host; }
/**//* return the port */
inline uint getPort () { return port; }
/**//* return the file */
inline char *getFile () { return file; }
/**//** Depth in the Site */
inline int8_t getDepth () { return depth; }
/**//* Set depth to max if we are at an entry point in the site
* try to find the ip addr
* answer false if forbidden by robots.txt, true otherwise */
bool initOK (url *from);
/**//** return the base of the url
* give means that you have to delete the string yourself
*/
url *giveBase ();
/**//** return a char * representation of the url
* give means that you have to delete the string yourself
*/
char *giveUrl ();
/**//** write the url in a buffer
* buf must be at least of size maxUrlSize
* returns the size of what has been written (not including '\0')
*/
int writeUrl (char *buf);
/**//* serialize the url for the Persistent Fifo */
char *serialize ();
/**//* very thread unsafe serialisation in a static buffer */
char *getUrl();
/**//* return a hashcode for the host of this url */
uint hostHashCode ();
/**//* return a hashcode for this url */
uint hashCode ();
#ifdef URL_TAGS
/**//* tag associated to this url */
uint tag;
#endif // URL_TAGS
#ifdef COOKIES
/**//* cookies associated with this page */
char *cookie;
void addCookie(char *header);
#else // COOKIES
inline void addCookie(char *header) {}
#endif // COOKIES
};
#endif // URL_H
三 代碼分析
url中的實(shí)現(xiàn)類主要是，創(chuàng)建url，其中創(chuàng)建規(guī)則如下：
http://www.hach.com/r/0343/ttt.html
則host為www.hach.com , file 為/r/0343/ttt.html
url的構(gòu)造函數(shù)即根據(jù) 上述規(guī)則構(gòu)建 url類。 若是含有base url 則新的url的file為 base->file + 新url 的file。
（2）另外url的hash函數(shù)即是 利用了 file 字符串和 host字符串。
/* return a hashcode for this url */
uint url::hashCode () {
unsigned int h=port;
unsigned int i=0;
while (host[i] != 0) {
h = 31*h + host[i];
i++;
}
i=0;
while (file[i] != 0) {
h = 31*h + file[i];
i++;
}
return h % hashSize;
}
(3) cookie的處理函數(shù)如下
若addCookie(char * head) 中的head字符串是以 set-cookie: 開始的，則將head之后的12個(gè)字符
添加到cookie變量中。
四綜上：
url 類中的成員變量，char * file ,char * host , port , cookie 能夠表示一個(gè)url。
并且url類中提供了解析函數(shù)，使用戶可以根據(jù)從網(wǎng)頁(yè)中爬取的url構(gòu)造url類對(duì)象。