Perl & Python编写CGI

浅浅的花香味﹌ 2022-08-27 01:13 204阅读 0赞

最近偶然玩了一下CGI，收集点资料写篇在这里留档。 现在想做HTTP Cache回归测试了，为了模拟不同的响应头及数据大小，就需要一个CGI按需要传回指定的响应头和内容，这是从老外的[测试页面][Link 1]学习到的经验。

CGI其实就是使用STDIN和环境变量作为输入,  STDOUT做为输出，依照Http协议生成对应的数据。

## 一. 数据输出 ##

数据输出遵循HTTP协议，分为四部分:

* * ***状态行 (Status Line):***

*     200 OK*

* * ***响应头(Response Headers):***

*     Content-Type: text/html*

*     Content-Length: 3072*

*   ****空白行(代表响应头已经列完, 只能包含回车和换行符):***

*   *

*   ****数据(文本或二进制数据):***

*   <html>xxxxxx</html>*

参考: [HTTP Protocol][]

状态行和响应头中每一行都必须以\\r\\n(回车及换行)结束。使用这个规则，以简单的文本组装就可以完成响应了。

只要注意二进制数据输出方法就可以了。 尝试设定Content-Length时可能没办法设定的准确，至少对于Apache Server是这样，不过不影响最后返回的数据量。

## 二. 数据输入 ##

CGI的数据输入主要是环境变量，下面这个链接有一串定义,

[CGI Environment Variables][]

<table> 
 <tbody> 
 <tr> 
 <td>Key</td> 
 <td>Value</td> 
 </tr> 
 <tr> 
 <td>DOCUMENT_ROOT</td> 
 <td>The root directory of your server</td> 
 </tr> 
 <tr> 
 <td>HTTP_COOKIE</td> 
 <td>The visitor's cookie, if one is set</td> 
 </tr> 
 <tr> 
 <td>HTTP_HOST</td> 
 <td>The hostname of the page being attempted</td> 
 </tr> 
 <tr> 
 <td>HTTP_REFERER</td> 
 <td>The URL of the page that called your program</td> 
 </tr> 
 <tr> 
 <td>HTTP_USER_AGENT</td> 
 <td>The browser type of the visitor</td> 
 </tr> 
 <tr> 
 <td>HTTPS</td> 
 <td>"on" if the program is being called through a secure server</td> 
 </tr> 
 <tr> 
 <td>PATH</td> 
 <td>The system path your server is running under</td> 
 </tr> 
 <tr> 
 <td>QUERY_STRING</td> 
 <td>The query string (see GET, below)</td> 
 </tr> 
 <tr> 
 <td>REMOTE_ADDR</td> 
 <td>The IP address of the visitor</td> 
 </tr> 
 <tr> 
 <td>REMOTE_HOST</td> 
 <td>The hostname of the visitor (if your server has reverse-name-lookups on; otherwise this is the IP address again)&nbsp;</td> 
 </tr> 
 <tr> 
 <td>REMOTE_PORT</td> 
 <td>The port the visitor is connected to on the web server</td> 
 </tr> 
 <tr> 
 <td>REMOTE_USER</td> 
 <td>The visitor's username (for .htaccess-protected pages)</td> 
 </tr> 
 <tr> 
 <td>REQUEST_METHOD</td> 
 <td>GET or POST</td> 
 </tr> 
 <tr> 
 <td>REQUEST_URI</td> 
 <td>The interpreted pathname of the requested document or CGI (relative to the document root)</td> 
 </tr> 
 <tr> 
 <td>SCRIPT_FILENAME</td> 
 <td>The full pathname of the current CGI</td> 
 </tr> 
 <tr> 
 <td>SCRIPT_NAME</td> 
 <td>The interpreted pathname of the current CGI (relative to the document root)</td> 
 </tr> 
 <tr> 
 <td>SERVER_ADMIN</td> 
 <td>The email address for your server's webmaster</td> 
 </tr> 
 <tr> 
 <td>SERVER_NAME</td> 
 <td>Your server's fully qualified domain name (e.g. <a href="http://www.cgi101.com/" rel="nofollow">www.cgi101.com</a>)</td> 
 </tr> 
 <tr> 
 <td>SERVER_PORT</td> 
 <td>The port number your server is listening on</td> 
 </tr> 
 <tr> 
 <td>SERVER_SOFTWARE</td> 
 <td>The server software you're using (e.g. Apache 1.3)&nbsp;</td> 
 </tr> 
 </tbody> 
</table>

当你需要CGI处理POST请求时，CGI就要使用STDIN做为输入来接收数据了。

在Perl里使用下面的代码读取:

use CGI;

my $cgi = CGI->new();

my %params = $cgi->Vars();

而在Python则是:

import cgi

cgi.FieldStorage()

参考: [Perl CGI Input Test][]

## 三. 开发语言 ##

可以写CGI的开发语言太多，下面附上两个分别使用Perl和Python编写的相同功能的CGI, 正好可以做个对比。

这两个脚本可以接收Query String, 然后返回不同的文件，数据大小，以及缓存相关的头信息，可以区分二进制数据和文本数据。

CGI脚本支持以Query String修改以下响应头:

content type,

cache control,

content length (只返回对应大小的数据),

last modified,

expires

下面是返回一个指定大小图片的例子:

/cgi/cache\_factory.pl?**type**=image&**size**=32&**last-modified**=Fri, 02 Apr 2014 02:34:06 GMT&**cache-control**=private,max-age=60&expires=Fri, 22 Apr 2014 02:34:06 GMT

代码很简单，可以做个参考。

### Perl版本 ###

#!/usr/bin/perl
 use strict;
 use warnings;
 use CGI;
 
 use constant BUFFER_SIZE => 4_096;
 use constant DATA_DIRECTORY => "/var/www/Cache";
 
 my $cgi = CGI->new();
 my %params = $cgi->Vars();
 
 &parserCommonHeaders;
 
 if(exists $params{'type'} && $params{'type'}=="image"){
 	&generateImageData;	 
 }
 else{
 	&generateTextData;
 }
 
 sub parserCommonHeaders{
 	if(exists $params{'cache-control'}){
 	 print 'Cache-Control:',$params{'cache-control'},"\r\n"; 
 	}
 
 	 if(exists $params{'last-modified'}){
 	 print 'Last-Modified:',$params{'last-modified'},"\r\n"; 
 	 }
 	 
 	 if(exists $params{'expires'}){
 	 print 'Expires:',$params{'expires'},"\r\n";
 	 }
 	 
 	 if(exists $params{'etag'}){
 	 print 'ETag:ea6186e11526ce1:0',"\r\n";
 	 }	
 }
 
 sub generateImageData{
 	my $buffer = "";
 
 	my $targetSize = 100*1024*1024;
 	if(exists $params{'size'} && $params{'size'}>0){
 		$targetSize = 1024*$params{'size'};
 		print "Content-length: $targetSize \r\n";
 	}
 
 	my $image = DATA_DIRECTORY .'/images/very_big.jpg';
 	my( $type ) = $image =~ /\.(\w+)$/;
 	$type eq "jpg" and $type == "jpeg";
 	
 	print $cgi->header( -type => "image/$type", -expires => "-1d" );
 	binmode STDOUT;
 	
 	local *IMAGE;
 	open IMAGE, $image or die "Cannot open file $image: $!";
 	
 	my $sentSize = 0;
 	while ( read( IMAGE, $buffer, BUFFER_SIZE ) ) {
 	 print $buffer;
 	 $sentSize += BUFFER_SIZE;
 	 
 	 if($sentSize>=$targetSize){
 	 	last;
 	 }
 	}
 	close IMAGE;
 }
 
 sub generateTextData{
 	my $startHeader = '<html><head><title>HTTP Cache Testing HTML</title></head><body>';
 	my $tailPart = '</body></html>';
 	
 	if(exists $params{'type'}){
 		print "Content-type:$params{'type'}\r\n";
 	}
 	else{
 		print "Content-type:text/html\r\n";
 	}
 
 	my $targetTextSize = 100*1024*1024;
 	if(exists $params{'size'} && $params{'size'}>0){
 		$targetTextSize = 1024*$params{'size'};
 		print "Content-length: $targetTextSize \r\n";
 	}
 	print "\r\n";
 	
 	$targetTextSize -= length($startHeader) + length($tailPart);
 	
 	print "$startHeader";
 	
 	my $filepath = DATA_DIRECTORY .'/files/big_text.txt';
 	
 	open(FILE, $filepath) or die $!;
 	my @lines = <FILE>;
 	close(FILE);
 	
 	foreach my $line (@lines) {
 	if( length($line)<=$targetTextSize ){
 		print $line;
 	}
 	else{
 		print substr($line,0,$targetTextSize);
 	}
 	
 	$targetTextSize -= length($line);
 	
 	if($targetTextSize<=0){
 		last;
 	}
 	}
 
 	print "$tailPart";
 }

### Python版本 ###

#!/usr/bin/python
 import os 
 import cgi
 import sys
 
 BUFFER_SIZE = 4096
 DATA_DIRECTORY = "/var/www/Cache"
 
 def parserCommonHeaders(formQuery):
 if('cache-control' in formQuery.keys()):
 print 'Cache-Control:',formQuery['cache-control'].value,"\r\n", 
 
 if('last-modified' in formQuery.keys()):
 print 'Last-Modified:',formQuery['last-modified'].value,"\r\n", 
 
 if('expires' in formQuery.keys()):
 print 'Expires:',formQuery['expires'].value,"\r\n",
 
 if('etag' in formQuery.keys()):
 print 'ETag:ea6186e11526ce1:0',"\r\n",
 
 
 def generateImageData(formQuery):
 targetSize = 100*1024*1024;
 if('size' in formQuery.keys()):
 targetSize = 1024*int(formQuery['size'].value)
 print "Content-length:",targetSize,"\r\n",
 
 image = DATA_DIRECTORY+'/images/very_big.jpg'
 print "Content-Type:image/jpeg\r\n",
 
 print 
 
 sentSize = 0
 f = open(image, 'rb')
 while True:
 data = f.read(4096)
 sentSize = sentSize + BUFFER_SIZE
 sys.stdout.write(data)
 if sentSize>targetSize:
 break
 
 sys.stdout.flush()
 close(f)
 
 
 def generateTextData(formQuery):
 startHeader = '<html><head><title>HTTP Cache Testing HTML</title></head><body>'
 tailPart = '</body></html>'
 
 targetTextSize = 2.3*1024*1024;
 if('size' in formQuery.keys()):
 targetTextSize = 1024*int(formQuery['size'].value)
 print "Content-length:",targetTextSize,"\r\n",
 
 if('type' in formQuery.keys()):
 print "Content-type: %s\r\n"%(formQuery['type'].value),
 else:
 print "Content-type: text/html\r\n",
 
 print
 
 print startHeader
 
 targetTextSize = targetTextSize - len(startHeader) - len(tailPart)
 
 filepath = DATA_DIRECTORY + '/files/big_text.txt'
 
 file = open(filepath)
 lines = file.readlines()
 file.close()
 
 for line in lines:
 if( len(line) <= targetTextSize ):
 print line
 else:
 print line[0:targetTextSize]
 
 targetTextSize = targetTextSize - len(line)
 
 if(targetTextSize<=0):
 break
 
 print tailPart
 
 if __name__ =="__main__":
 formQuery = cgi.FieldStorage() #os.environ['QUERY_STRING']
 
 parserCommonHeaders(formQuery)
 
 if('type' in formQuery.keys() and formQuery['type'].value=="image"):
 generateImageData(formQuery) 
 else:
 generateTextData(formQuery)

## 四. 服务器 ##

服务器端使用Apache Server+WANem, 配合CGI完成灵活的需求，开通SFTP端口供相关同学编辑，方便共享测试用例。

.-----.   .-------.

| CGI |   | WANem |

'-----'---'-------'

| Apache Server   |

'-----------------'

SFTP & HTTP

.------------------.

| Web Page Editor  |

| and Browser      |

'------------------'

配有WANem最大的好处就是可以根据需求进网络状态调整, 甚至可以用类似下面的方式在测试用例动态调整。

## 参考: ##

[Apache配置][Apache] (不要忘记给CGI脚本加上可执行权限)

[CGI Programming 101 (Perl)][CGI Programming 101 _Perl]

[Debugging CGI Programs (Perl)][Debugging CGI Programs _Perl]

[Python CGI编程][Python CGI]

[Python CGI Debugging][]

[Link 1]: http://stevesouders.com/tests/redirects/
[HTTP Protocol]: http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Response_message
[CGI Environment Variables]: http://www.cgi101.com/book/ch3/text.html
[Perl CGI Input Test]: http://www.sp.uconn.edu/~jrifkin/cgi-bin/argtest/
[Apache]: http://httpd.apache.org/docs/2.2/howto/cgi.html
[CGI Programming 101 _Perl]: http://www.cgi101.com/book/
[Debugging CGI Programs _Perl]: http://101.lv/learn/CGIperl/ch13.htm
[Python CGI]: http://www.w3cschool.cc/python/python-cgi.html
[Python CGI Debugging]: http://webpython.codepoint.net/cgi_debugging