Detecting rapid access requests ... - geek/talk: Signature-free discourse for serious web publishers

JohnKelly · 08-24-2001, 06:49 PM

Hi, I have a problem and am in search of a script that may help. The problem is whenever a unruly spider or cable/dsl visitor using a link indexing script, requests way to many pages at once, like 100's per minute, one of my servers runs out of RAM and all that server's services start locking up and if not caught in time eventually the server requires a hard reboot as I can't even get to the command line.

I have been told to add more RAM and while that may be one obvious solution, I would perfer to attack the true cause of the problem. For example a script that tails my access_log file and detects whenever an IP address makes X amount of requests within X seconds after which it emails me and/or issues a command.

Is anyone familiar with a script such as this. I found one called
SWATCH but I think it is designed to detect regular expressions in log files rather than number of requests over a time level. Any help appreciated!

JakeJeck · 08-30-2001, 10:46 AM

I just bought 256 Megs of ram for $40 at Best Buy. IMO buying more RAM will be a much cheaper solution than any script that requires programming. More RAM would also potentially speed up everything your server does whereas a script would just prevent this one problem.

LastActionHero · 08-30-2001, 12:50 PM

I think using .htaccess or something in the apache conf file you can limit the number of simultaneous connections from an IP to an integer value.

I'm not sure.

08-24-2001, 06:49 PM	#1
JohnKelly Registered User Join Date: Aug 2001 Posts: 53	Detecting rapid access requests ... Hi, I have a problem and am in search of a script that may help. The problem is whenever a unruly spider or cable/dsl visitor using a link indexing script, requests way to many pages at once, like 100's per minute, one of my servers runs out of RAM and all that server's services start locking up and if not caught in time eventually the server requires a hard reboot as I can't even get to the command line. I have been told to add more RAM and while that may be one obvious solution, I would perfer to attack the true cause of the problem. For example a script that tails my access_log file and detects whenever an IP address makes X amount of requests within X seconds after which it emails me and/or issues a command. Is anyone familiar with a script such as this. I found one called SWATCH but I think it is designed to detect regular expressions in log files rather than number of requests over a time level. Any help appreciated!

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Can't access some websites ?	dotsplash	Web Design and Webmaster Issues	4	10-29-2001 06:30 AM
only allow access to a page from 1 url	Bassmaster	Web Development - Programming / Coding	4	08-02-2001 02:28 AM
ISPs Pay Content Providers for Access	AK	Making Money with CPC and/or CPM Programs	2	07-13-2001 09:26 PM
Wireless laptop internet access	demae	geek/yak	3	03-11-2001 12:38 AM
Database access w/ CGI	Ken-	Web Design and Webmaster Issues	13	05-24-2000 05:59 PM

08-30-2001, 10:46 AM	#2
JakeJeck Registered User Join Date: Mar 2000 Location: Milwaukee WI Posts: 338	I just bought 256 Megs of ram for $40 at Best Buy. IMO buying more RAM will be a much cheaper solution than any script that requires programming. More RAM would also potentially speed up everything your server does whereas a script would just prevent this one problem.

08-30-2001, 12:50 PM	#3
LastActionHero Registered User Join Date: Jun 2001 Location: New Delhi, India Posts: 615	I think using .htaccess or something in the apache conf file you can limit the number of simultaneous connections from an IP to an integer value. I'm not sure.