今天在提取抓取到的網(wǎng)頁(yè)內(nèi)容的正文時(shí)候,發(fā)現(xiàn)結(jié)果老是不完整,開(kāi)始以為是提取不正確,然后去一步步分析提取結(jié)果,發(fā)現(xiàn)沒(méi)問(wèn)題,最后才發(fā)現(xiàn)是編碼轉(zhuǎn)換的環(huán)節(jié)的問(wèn)題。我開(kāi)始是直接使用iconv函數(shù),
$txtContent = iconv("utf-8",'GBK',$txtContent);
utf-8直接轉(zhuǎn)gbk,這樣問(wèn)題就來(lái)了,當(dāng)有些字符無(wú)法轉(zhuǎn)換的時(shí)候就從此處斷開(kāi),導(dǎo)致內(nèi)容不完整。后來(lái)又重新查手冊(cè),才發(fā)現(xiàn)iconv還有兩個(gè)可選的輔助參數(shù):TRANSLIT和IGNORE ,(其中IGNORE 就是說(shuō)遇到無(wú)法轉(zhuǎn)換的就跳過(guò))。 其實(shí)也怪自己太粗心,剛開(kāi)始一看iconv函數(shù)參數(shù)就三個(gè),而且例子中也沒(méi)有特別的,就懶得去看英文解釋了,直接使用,直到出現(xiàn)了問(wèn)題。
Description
string iconv ( string in_charset, string out_charset, string str )
Performs a character set conversion on the string str from in_charset to out_charset. Returns the converted string or FALSE on failure.
If you append the string //TRANSLIT to out_charset transliteration is activated. This means that when a character can't be represented in the target charset, it can be approximated through one or several similarly looking characters. If you append the string //IGNORE, characters that cannot be represented in the target charset are silently discarded. Otherwise, str is cut from the first illegal character.
posted on 2012-09-03 15:46
小果子 閱讀(237)
評(píng)論(0) 編輯 收藏 引用 所屬分類:
學(xué)習(xí)筆記