URGENT READ ME - Unacceptable Forum Activity
Moderators: Chuck, Mustang380gal, Stryker74, JediSkipdogg, Jake, evan price, djmac1964, Derek D., Bryan Torok, schmieg, sabalo, oldmic, OldManTod, NavyChief, MrMagoo, Morne, MeanStreaker, jgarvas, JustaShooter, 3FULLMAGS+1, Coordinators
- jgarvas
- OFCC Director
- Posts: 3164
- Joined: Sun Jun 26, 2005 10:35 pm
- Location: Northern Summit County
- Contact:
URGENT READ ME - Unacceptable Forum Activity
It has been brought to my attention that users of the forums have discussed using the open source tool 'wget' and windows variants of this tool to mass-download the forums to their local computers for off-line reading and archival purposes.
Effective immediately, anyone caught loading a substantially large number of pages at a rapid pace will have their account suspended regardless of what software they use to do so.
This recent activity has put a substantial burden on the forums and has quite possible caused all of our "You May Not Use The Search Function" errors as individual users consume substantial memory and database resources to rapidly download significant quantities of forums activity.
If we determine that users continue to do this their accounts will be terminated indefinitely. If the problem persists, we will configure the forums to lock out those people who load more than a reasonable amount of pages in a specific amount of time.
We appreciate the fact that many of you find the forums so useful that you wish to archive them for personal use, but we must protect the integrity of the entire system
Effective immediately, anyone caught loading a substantially large number of pages at a rapid pace will have their account suspended regardless of what software they use to do so.
This recent activity has put a substantial burden on the forums and has quite possible caused all of our "You May Not Use The Search Function" errors as individual users consume substantial memory and database resources to rapidly download significant quantities of forums activity.
If we determine that users continue to do this their accounts will be terminated indefinitely. If the problem persists, we will configure the forums to lock out those people who load more than a reasonable amount of pages in a specific amount of time.
We appreciate the fact that many of you find the forums so useful that you wish to archive them for personal use, but we must protect the integrity of the entire system
Jeff Garvas, President
Ohioans For Concealed Carry
Contrary to a popular belief when I brag about OFCC accomplishments I'm not looking for your thank you or personal recognition. I'd much prefer you send me an email telling me when you are going to get involved in doing what I've been doing since 1999. We are only as effective as we make ourselves. We need the next generation of OFCC to step to the plate.
Is that you?
To Contact Me: Use This Form and pick my name.
Ohioans For Concealed Carry
Contrary to a popular belief when I brag about OFCC accomplishments I'm not looking for your thank you or personal recognition. I'd much prefer you send me an email telling me when you are going to get involved in doing what I've been doing since 1999. We are only as effective as we make ourselves. We need the next generation of OFCC to step to the plate.
Is that you?
To Contact Me: Use This Form and pick my name.
-
- Posts: 1398
- Joined: Wed Dec 06, 2006 3:43 pm
- Location: toledo ohio
- Contact:
Re: URGENT READ ME - Unacceptable Forum Activity
i have not done this but i have considered archiving the forums on my hard drive. is it possible that someone could do this and post the download on the website? that may solve this problem
NRA Certified Basic Pistol Instructor
PM for class info
http://www.facebook.com/pages/Patriots- ... 1429217356" onclick="window.open(this.href);return false;
PM for class info
http://www.facebook.com/pages/Patriots- ... 1429217356" onclick="window.open(this.href);return false;
-
- Posts: 9557
- Joined: Mon Jul 04, 2005 1:36 am
- Location: Youngstown OH
Re: URGENT READ ME - Unacceptable Forum Activity
Jeff:
Won't IE7's "Work Offline" click do something like this? (I think it's "page level" only, but I'm not sure how deep it'll go.)
Regards,
Won't IE7's "Work Offline" click do something like this? (I think it's "page level" only, but I'm not sure how deep it'll go.)
Regards,
Stu.
(Why write a quick note when you can write a novel?)
(Why do those who claim to wish to protect me feel that the best way to do that is to disarm me?)
יזכר לא עד פעם
(Why write a quick note when you can write a novel?)
(Why do those who claim to wish to protect me feel that the best way to do that is to disarm me?)
יזכר לא עד פעם
-
- OFCC Member
- Posts: 10911
- Joined: Mon Jul 04, 2005 6:41 pm
- Location: Mercer County, Ohio - what is yours?
Re: URGENT READ ME - Unacceptable Forum Activity
It is doing it again!
Sorry but you cannot use search at this time. Please try again in a few minutes.
Abandon ye all HOPE!
- NordicRX8
- Posts: 2154
- Joined: Sat Apr 21, 2007 10:40 pm
- Location: Lost In Middle America, Ohio
Re: URGENT READ ME - Unacceptable Forum Activity
Would opening twenty tabs put the system under the same strain? I normally open each sub forum in its own tab, and then open each topic of interest in its own tab... I sometime have over 2 dozen tabs open at the same time.
Am I contributing to this error?
Let me know and I can stop doing this.
Am I contributing to this error?
Let me know and I can stop doing this.
Suppose the Second amendment said "A well-educated electorate being necessary for self-governance in a free state, the right of the people to keep and read books shall not be infringed." Is there anyone who would suggest that means only registered voters have a right to read? – Robert Levy
A gun in the hands of a bad man is a very dangerous thing. A gun in the hands of a good person is no danger to anyone except the bad guys... - Charlton Heston
A gun in the hands of a bad man is a very dangerous thing. A gun in the hands of a good person is no danger to anyone except the bad guys... - Charlton Heston
-
- Posts: 9557
- Joined: Mon Jul 04, 2005 1:36 am
- Location: Youngstown OH
Re: URGENT READ ME - Unacceptable Forum Activity
Nordic:
I don't remember IE, but FireFox will let you indicate a group in your "favorites" and open all of the members of the group automatically. If you're doing that, you may have a spike for a minute or so, if that long. I wouldn't worry about it - especially if you're opening them manually.
"Open" doesn't count. Once the page is loaded most of the "action" is inside your browser unless you do something to agitate the server. You can't do that fast enough to bother anything unless you leave a brick on the "Enter" key....
Regards,
I don't remember IE, but FireFox will let you indicate a group in your "favorites" and open all of the members of the group automatically. If you're doing that, you may have a spike for a minute or so, if that long. I wouldn't worry about it - especially if you're opening them manually.
"Open" doesn't count. Once the page is loaded most of the "action" is inside your browser unless you do something to agitate the server. You can't do that fast enough to bother anything unless you leave a brick on the "Enter" key....
Regards,
Stu.
(Why write a quick note when you can write a novel?)
(Why do those who claim to wish to protect me feel that the best way to do that is to disarm me?)
יזכר לא עד פעם
(Why write a quick note when you can write a novel?)
(Why do those who claim to wish to protect me feel that the best way to do that is to disarm me?)
יזכר לא עד פעם
-
- Posts: 1383
- Joined: Sun Nov 05, 2006 9:43 am
- Location: Middletown, Ohio
Re: URGENT READ ME - Unacceptable Forum Activity
Dang , I had you scheduled for this weekend...................I got to earn my "forum leech" title some way
I have discussed doing other forums but I haven't done this one..................
I have discussed doing other forums but I haven't done this one..................
"Our government... teaches the whole people by its example. If the government becomes the lawbreaker, it breeds contempt for law; it invites every man to become a law unto himself; it invites anarchy."
- cmarks03
- Posts: 348
- Joined: Mon Oct 06, 2008 9:21 pm
- Location: Cuyahoga County
Re: URGENT READ ME - Unacceptable Forum Activity
Not sure how the server that runs these forums is configured, but I'm sure that there is a way to monitor the access logs for crawlers (either a crawler like Google uses or `wget` in recursive mode). Knowing the open source community, there's probably a ton of configuration options for that kind of software (people mentioned multiple tabs, which I'm guilty of doing a bit of, so it would need look for a constant stream of requests). But once you have the info, you can either ban a forum user or ban an IP address (which I personally recommend against unless the ban is temporary).
It would all be on the server side though, and might require more control than our webmaster(s) have.
It would all be on the server side though, and might require more control than our webmaster(s) have.
- Chris
"I are a engineer"
"I'm not fat - I'm grossly overdesigned" (Morne)
"People can see your gun. Just be sure to reserve the view of the muzzle for the really bad ones." (muxtech)
Though I do have Progressive insurance, IANAL and I didn't stay at a Holiday Inn Express last night. This is my $.02; remember you paid less than that for it.
"I are a engineer"
"I'm not fat - I'm grossly overdesigned" (Morne)
"People can see your gun. Just be sure to reserve the view of the muzzle for the really bad ones." (muxtech)
Though I do have Progressive insurance, IANAL and I didn't stay at a Holiday Inn Express last night. This is my $.02; remember you paid less than that for it.
-
- Posts: 189
- Joined: Sun Nov 16, 2008 11:36 pm
- Location: Toledo, OH
Re: URGENT READ ME - Unacceptable Forum Activity
Your typical browsing no matter how many tabs you have open will not affect the site, the wget app is designed to (in the simplest form of explaining it) download an offline copy of the entire site by crawling every aspect of the site much like a search engine spider crawls the site.
- jgarvas
- OFCC Director
- Posts: 3164
- Joined: Sun Jun 26, 2005 10:35 pm
- Location: Northern Summit County
- Contact:
Re: URGENT READ ME - Unacceptable Forum Activity
We have full access and the environment is apache, and for the most part I know what I'm doing but I focus more on Cisco lately than unix/lamp. We've identified the culprits (and a very aggressive spider) and have rectified the problem. If you're using the forums to read them live (even opening a bunch of tabs) I really don't see that comparing to the volume of rapid data an aggressive depth wget would generate.cmarks03 wrote:Not sure how the server that runs these forums is configured, but I'm sure that there is a way to monitor the access logs for crawlers (either a crawler like Google uses or `wget` in recursive mode). Knowing the open source community, there's probably a ton of configuration options for that kind of software (people mentioned multiple tabs, which I'm guilty of doing a bit of, so it would need look for a constant stream of requests). But once you have the info, you can either ban a forum user or ban an IP address (which I personally recommend against unless the ban is temporary).
It would all be on the server side though, and might require more control than our webmaster(s) have.
The problem for us isn't bandwidth at all. The problem is resource related. Since everything is database driven (every post) a very large and rapid demand for post after post after post causes the system to become taxed -- and quite honestly we didn't design it to support that kind of behavior because that isn't our target audience and it would be cost prohibitive.
If the issue keeps up, we'll install checks and balances that react if they see a specific session requesting too many sites in a very rapid succession instead of the more broad shutting down the forums, but recent changes seem to have nipped this issue for now.
I should clarify, if someone is reading the forums in a manner that is hurting us, we won't kick you out and shut off your account and throw away the key. We'll try to talk to you and determine what you're doing.
We will however take more aggressive action with people who intentionally and/or maliciously harm the forums for their own personal satisfaction and refuse to work with us if their access is detrimental to the system and/or it's resources.
Jeff Garvas, President
Ohioans For Concealed Carry
Contrary to a popular belief when I brag about OFCC accomplishments I'm not looking for your thank you or personal recognition. I'd much prefer you send me an email telling me when you are going to get involved in doing what I've been doing since 1999. We are only as effective as we make ourselves. We need the next generation of OFCC to step to the plate.
Is that you?
To Contact Me: Use This Form and pick my name.
Ohioans For Concealed Carry
Contrary to a popular belief when I brag about OFCC accomplishments I'm not looking for your thank you or personal recognition. I'd much prefer you send me an email telling me when you are going to get involved in doing what I've been doing since 1999. We are only as effective as we make ourselves. We need the next generation of OFCC to step to the plate.
Is that you?
To Contact Me: Use This Form and pick my name.
- shdwfx
- Posts: 187
- Joined: Sat Feb 02, 2008 11:48 pm
- Location: NW Ohio
Re: URGENT READ ME - Unacceptable Forum Activity
If someone is using wget or another archival software, it's unlikely that client is bothering with the step of logging in (or in wget's case, spoofing the login with a browser cookie file).
So, the only identifying info available from such harmful activity is going to be an IP address.
You might then scan logs to see which forum ID(s) hail from those IP addresses, but it's not guaranteed to be 1 to 1 due to Network Address Translation (NAT).
So, the only identifying info available from such harmful activity is going to be an IP address.
You might then scan logs to see which forum ID(s) hail from those IP addresses, but it's not guaranteed to be 1 to 1 due to Network Address Translation (NAT).
-
- Posts: 268
- Joined: Fri Jun 20, 2008 1:39 am
- Location: Lake County
- Contact:
Re: URGENT READ ME - Unacceptable Forum Activity
Is this why I get this message when I search? Well, here comes a bucket of stupid questions from me that would have been answered through the search function. Sorry in advance.Cruiser wrote:It is doing it again!Sorry but you cannot use search at this time. Please try again in a few minutes.
Heavy is good, heavy is reliable. If it doesn't work you can always hit him with it.
-Boris 'The Blade' Yurinov
-Boris 'The Blade' Yurinov