海运的博客

1.使用PHP GeoIP扩展，功能不全，如不能查询AS号：

yum install php-pecl-geoip
#下载最新geo数据库
cd /usr/share/GeoIP/
wget http://www.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
gzip -d GeoIP.dat.gz

<?php
echo geoip_country_code_by_name('www.haiyun.me') . "\n";
?>

2.使用GeoIP官方提供的PHP API，不可和上扩展共存，函数冲突。

git clone https://github.com/maxmind/geoip-api-php.git

<?php
include("./src/geoip.inc");
$gi = geoip_open("/usr/share/GeoIP/GeoIP.dat", GEOIP_STANDARD);
echo geoip_country_code_by_name($gi, "www.haiyun.me") . "\n";
?>

GeoIP数据库下载：http://dev.maxmind.com/geoip/legacy/geolite/

在使用PHP pcre_match时遇到错误：

Compilation failed: this version of PCRE is not compiled with PCRE_UTF8 support at offset 0

pcretest -C
Compiled with UTF-8 not support

下载对应版本src rpm重新编译pcre：

yum install rpm-build
wget http://vault.centos.org/6.4/os/Source/SPackages/pcre-7.8-6.el6.src.rpm
rpmbuild --rebuild --with enable-utf8 pcre-7.8-6.el6.src.rpm

重新安装编译的软件包：

rpm -iv --replacepkgs /root/rpmbuild/RPMS/i386/pcre-7.8-6.el6.i386.rpm 

英文：

function tourl($str) {
  $str = preg_replace('/[^a-zA-Z0-9]+/', '-', $str);
  $str = trim($str, '-');
  return strtolower($str);
}

包含特殊字符：

function strtourl($str) {
  $str = htmlentities($str, ENT_QUOTES, "UTF-8");
  $str = preg_replace('/&([a-z]{1,2})(?:acute|lig|grave|ring|tilde|uml|cedil|caron);/i','\1',$str);
  $str = html_entity_decode($str, ENT_QUOTES, "UTF-8");
  $str = preg_replace('/[^a-z0-9]+/i', '-', $str);
  return strtolower($str);
}

1.phpQuery

<?php
   require("phpQuery-onefile.php");
   phpQuery::newDocumentFile('https://www.haiyun.me/archives.html'); 

   //foreach (pq('body .main li')->find('a') as $a) {
   foreach (pq('body .main li a') as $a) {
      $hrefs[] = pq($a)->attr('href');
      $hrefs[] = pq($a)->text();
   }
   print_r($hrefs);
?>

记得释放内存：

phpQuery::$documents = array();
phpQuery::unloadDocuments();

2.基于tidy的HtmlParserModel，可解析不正规的HTML页面：

yum install php-tidy
git clone https://github.com/bupt1987/HtmlParserModel.git

<?php
   include_once "HtmlParserModel.php";
   $html = file_get_contents('http://www.amazon.com/s/node=3564986011');
   $html_dom = new HtmlParserModel($html);
   $p_array = $html_dom->find('a.title');
   foreach ($p_array as $p){
      echo $p->getPlainText();
   }
?>

<?php  
   $start = microtime ( true ); 
   function convert($size)
   {
      $unit=array('b','kb','mb','gb','tb','pb');
      return @round($size/pow(1024,($i=floor(log($size,1024)))),2).$unit[$i];
   }

   function strtr_words($str)  
   {  
      $words=array();  
      $content = file_get_contents('t.txt');
      $content = preg_split('/\r\n/', $content, -1, PREG_SPLIT_NO_EMPTY); //分割字符串为数组
      foreach($content as $key => $value)  
      {  
         //if ($key != '')  
         if (!empty($key))  
         {  
            $str_data = explode(',', $value); //分割同义词为k-v数组
            $str = str_replace($str_data[0], $str_data[1] , $str);
            //$words+=array("$str_data[0]"=>"$str_data[1]");  
         }  
      }  
      return $str;
      //return strtr($str,$words);
   }  
   echo strtr_words('村庄,一眼望不到边际')."\n";  
   echo convert(memory_get_usage(true))."\n"; 
   $end = microtime ( true ); 
   echo "脚本执行时间".($end - $start)."\n";  
?>

14.12.22更新：
此方法词库太大时效率很差，可以将词库以key->vale方式加载到内存，文章分词遍历再替换。