Can I run this scrip under VirtualMin ?

Virtualmin
1

Hi,

Firstly, I apologize for my ignorance on this subject !

I have been looking for ways to get web-site screen shots from urls

to place on one of of my websites.

I found a neat solution but it needs a Windows server. My server is
running CentOS Linux 5.4 with Virtualmin version 3.75 Pro.

Then I found this script but I am not sure how to run it :frowning:

Is the script in pearl ? I write in php so I don’t know much about this
or how I can check if I have the required s/w on my server.

Here is the script:

I used the “code” tags so I hope it will format OK on this page

#!/bin/sh

A script to take screenshots of webpages.

Example usage: webthumb.sh << END

http://www.mozilla.org mozilla.png mozilla_thmb.png

http://www.imagemagick.org imagemagick.png imagemagick_thmb.png

http://www.uk.research.att.com/archive/vnc/xvnc.html xvnc.png xvnc_thmb.png

END

Requires:

* ImageMagick

* Mozilla

* Xvnc (as Virtual X server), you might also fire up X.org for that purpose

Author: Henryk Gerlach (2005-01-01)

No warranty, use at own risk.

This script is Public Domain. All the hard work is done third party software

anyway.

#BUGS:

Mozilla:

* -geometry does not work with mozilla

* if mozilla already runs under the userid it might not start up.

* if mozilla needs to be configured for the userid running it, the script

won’t work

BROWSER=mozilla

#The window title of the browser after startup, so we can grab it’s window id.
#If all fails, take root to grab the whole desktop
BROWSERID=“Mozilla”
#BROWSERID=“root”

DISPLAY=":1"
#Use virtual X server
VIRTUAL_X=“Xvnc -depth 16 -localhost -geometry 1024x768 $DISPLAY”

#cut of browser decoration for 800x600 resolution
CROP=“800x490+0+90”
THUMBSIZE=“120x90”

INITTIME=5 #time to wait for the Xserver and the browser to start up
LOADTIME=4 #time to wait for a page to load, take shot after this time

######################################################

#init
if test “$VIRTUAL_X” != “”; then
$VIRTUAL_X &
fi
export DISPLAY
sleep $INITTIME
$BROWSER &
sleep $INITTIME
if test “$BROWSERID” != “root”; then
BROWSERID=xwininfo -name "$BROWSERID" | grep "Window id" | cut -d " " -f 4
fi

loop

while read URL BIG THUMB; do
$BROWSER -remote “openurl($URL)”
sleep $LOADTIME
import -window “$BROWSERID” $BIG
if test “$CROP” != “”; then
mogrify -crop $CROP “$BIG”
fi
convert -size “$THUMBSIZE” “$BIG” -resize “$THUMBSIZE” “$THUMB”
done

#cleanup
#this is ugly
if test “$VIRTUAL_X” != “”; then
killall Xvnc
else
killall $BROWSER
fi

Can anyone help me know how to use this and check for the required s/w ?

Thanks
Dave.

2

Howdy,

You can tell what sort of script it is by the first line
 the “#!/bin/sh” line means it uses “/bin/sh”, or a shell, to interpret the code.

However, the script above is meant to run on a desktop rather than a server. It requires an X server and a GUI-based browser, according to what I’m seeing there.

That may mean that the tool you found that requires Windows is okay
 if it’s just using your web browser to remotely access your server, Windows should work perfectly fine for that.

-Eric

3

Thanks for your input,

The problem is, I need to run this on my server because it needs to serve the iages on demand to my website.

To give you an idea of what I mean, I’ll give you this link:

http://www.expert-world.com/sys/cb_detail.php?pd=AFFILORAMA

You see the image there - that screen shot ?

That is not stored anywhere but is captured on demand by server side software and thats what I need to do.

The script is supposed to what I need to run on unix isn’t VIRTUAL_X something that runs on unix?

In another forum I saw this:

Xvfb might be a better option than Xvnc. :wink:
VIRTUAL_X=“Xvnc -depth 16 -localhost -geometry 1024x768 $DISPLAY”
VIRTUAL_X=“Xvfb $DISPLAY”

I don’t fully understand this - Is a an X server a Windows server then ?

I “googled” the phrase and read a few pages. this is the closest thing I could find:

So are you saying that I still need to use a windows server to make this work ?

My home pc and it’s connection is not stable enough to leave running all the time for a web appliation (also it is close to our bedroom and the fan noise would be too much anyway !! )

Is there no way I can run this on my existing server ?

4

Howdy,

An “X Server” is what provides a GUI on Linux/UNIX systems. Those are typically not installed for servers, but only for desktops.

As far as the above script goes – I’m just not sure I see that as being the robust option you’re looking for :slight_smile:

Running both an X Server and Mozilla on your server, and querying Mozilla to take pictures of certain websites on the fly, seems well, problematic
 I’m inclined to think that there’s a better way, even if I’m not sure what that is at the moment :slight_smile:

Doing a little digging, I see that there’s some online services who do what you’re after:

http://sharebrain.info/webdesign/the-best-free-screenshot-services/2366/

http://www.webshotspro.com/

There’s bound to be some software you can install that does all this too
 for example, a little Googling brought up this project:

http://khtml2png.sourceforge.net/

That’s a bit older, they haven’t released anything since 2008, but there many be other similar options out there.

-Eric

5

Thanks for the input.

I tried the free services but they are not reliable enough, either they have 20 second plus delays or can not find the urls ( eg www.ford.com !!! )

So I have been looking up how to run it on my server.

I have spent a hour or so reading through a lot of stuff and I found this :

Here is the url but I quote some of it below:

http://vertito.blogspot.com/2008/02/howto-thumbnail-website-from-linux.html

Blogging from Wordpress platform enables a blogger to incorporate and activate snapshot plugin where a blogger create on-demand URL snap shots of their websites.

If you are looking for HTML site thumbnailer software available from Linux command lin, read on.

Here’s how to create a thumbnail images of URL sites from linux terminal.

Gnome Web Photo contains a thumbnailer that will be used by GNOME applications, including the file manager, to generate screenshots of web pages.

Gnome-Web-Photo Thumbnailer Fedora Installation

To generate thumbnail images using gnome-web-photo

yum -y install gnome-web-photo

Sample Usage:

Creating thumbnails image of a website or URL site.

gnome-web-photo -t 0 --mode=thumbnail -w 1152 -s 128 http://www.yahoo.com yahoo.jpg

gnome-web-photo -t 0 --mode=thumbnail -w 1152 -s 256 http://www.yahoo.com yahoo2.jpg

gnome-web-photo -t 0 --mode=thumbnail -w 512 -s 256 http://www.google.com google.jpg

After a while, try to view the thumbnail images

gthumb google.jpg

gview yahoo.jpg

Now I am not sure how I am going to run this from my php script though.

Do you think this is something I am going to be able to use ?

BTW
I also came across this thread on a forum:

I would like to execute gnome-web-photo program on CentOS server with PHP but it requires X to run. Do you have any ideas how I can do this ?

You could maybe have an X instance running and then call the gnome application with -display x:x (using whatever display would be appropriate).

Or maybe use an X server designed to be headless? Xvfb comes to mind but I have no prior experience with.

You can install “VNC server” on your server and connect it via VNC client.It will give you X-Windows access to your server and you can easily run your gnome-web-photo application.

Does that last forum answer make sense to you ?

Thanks again.

6

Well, the thing to remember, I think, is that this is a difficult problem to solve :slight_smile:

All these services exist because it’s a pain in the butt to do it right. And then, even the dedicated services struggle sometimes


The website you shared above that does what you’re after, over at expert-world.com – they’re using an online service for this called snapcasa.com, it appears.

As far as Gnome Web Photo goes – that has similar drawbacks to the initial script you posted
 it requires running a GUI on your server, and using GUI-based tools to perform the task. I think they were moreso designed for occasional use, rather than being run from within a website every time it gets a hit. I feel similarly about the forum response.

Sure, it’d probably work, for the most part, but I wouldn’t do that on a server I ran :slight_smile: It wouldn’t take many hits before you were completely out of resources.

If you aren’t sure how to go about all this, and you consider it an important part of your business, you might consider hiring a coder to develop a tool to handle this for you.

-Eric

7

Actually, of the options I’ve seen so far, I’d probably start by looking into khtml2png.

And if that didn’t work, I’d probably go back to the drawing board and see what all my options were, and deciding how much I was willing to spend to get someone to help out :slight_smile:

-Eric

8

Hrm, upon further exploration – it would seem that khtml2png requires an X Server too.

I think that first, I’d really try to make absolute certain that I couldn’t bear to use one of the online services who provide this function. I’d go through, check them all out, and see if one were tolerably fast – remembering that what you’re after is a time consuming task.

And only then would I consider trying to run it myself, using something like khtml2png. But personally, I’m not sure that I’d run that on a server I managed unless I had a crazy overabundance of available resources :slight_smile:

-Eric

9

Thanks for your advice, I really appreciate. :slight_smile:

One final option.

Again this requires a Windows server though:

It’s the php imagegrabwindow()

In the Manual it gives this code example:

<?php $browser = new COM("InternetExplorer.Application"); $handle = $browser->HWND; $browser->Visible = true; $browser->Navigate("http://www.libgd.org");

/* Still working? */
while ($browser->Busy) {
com_message_pump(4000);
}
$im = imagegrabwindow($handle, 0);
$browser->Quit();
imagepng($im, “iesnap.png”);
imagedestroy($im);
?>

It looked great until I read the first comment.

comment:

This function was painfully slow when I was testing it on my machine. It took about 2 or 3 seconds for it to return an image. It also fails to work if the Apache service doesn’t have access to “Interact with the desktop”.

If I were to set up a Windows shared server (cost about $5 month) would it have access to “Interact with the desktop” ?

BTW
When googling for the script I was looking out for any “guru.com” or “elance” type results that may point me towards a script writer but I didn’t notice any at all.

10

Hey,

I’ve been reading all the responses so far, which are definitely good ones.

Here’s something perhaps you can also consider, which sort of combines a few ideas but should also assist you in rendering time significantly.

First off, I’m of the mindset that unless you can do something better, there’s simply no reason to reinvent the wheel if a solid solution already exists.

Having said that
 Here’s what I’d suggest.

Use a service LIKE snapcasa.com but NOT directly.

What I mean is, DON’T link directly to their images using an image tag like they mention on their site. (unless for some reason the below is against their terms of use)

Instead, create a simple PHP script which retrieves the thumbnails and saves them in a local cache location on your server. While the thumbnails are being created, simply have your script show a “temporarily unavailable” image, and once they are available have it automatically show the thumbnail.

By doing this, you’ll be able to instantly show your visitors the a thumbnail, while allowing your script to get them behind the scenes.

After building dozens of scripts which work with remote services, I’ve found this technique is fairly efficient for things like what you’re trying to do. It saves you having to reinvent the process, and lets you get back to building an excellent website.

If you’d like a sample script which can do the above mentioned, please let me know.

*** It’s important to ensure that using the above technique is within the acceptable usage policy of the provider in question. ***

-Peter

11

David,

You can safely remove your email from the forum if you wish now. I’ve jotted it down, and will be in contact with you shortly regarding your request.

-Peter

12

Thanks Peter
I appreciate the input and I think that’s a sensible way to
approach this problem.

I will gladly take up your offer and resist re-inventing your wheel as well :slight_smile:

If you can send the script to the following email, that will be a great help and will certainly get me off to a good start :slight_smile:

*removed * I will edit this address out after I receive it.

And, thanks again.

David.