A Windows XP help forum. PCbanter

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

Go Back   Home » PCbanter forum » Microsoft Windows 7 » Windows 7 Forum
Site Map Home Register Authors List Search Today's Posts Mark Forums Read Web Partners

Wget question which only an expert can answer (obtain vpn config files from vpngate.net)



 
 
Thread Tools Rating: Thread Rating: 2 votes, 5.00 average. Display Modes
  #1  
Old December 25th 16, 04:41 PM posted to alt.comp.os.windows-10,alt.windows7.general
Hazuki Nakamura
external usenet poster
 
Posts: 78
Default Wget question which only an expert can answer (obtain vpn config files from vpngate.net)

The expert only question is how can we get this wget command to work?

c:\ wget --wait=3 --random-wait -r -I /en,/common -A
do_openvpn.aspx,openvpn_download.aspx\* -nc -nd http://www.vpngate.net/en/
-e robots=off

I must warn any respondents that this wget syntax question is so difficult
to answer that I do not expect any answer (except perhaps from someone who
is so proficient in wget syntax that they can code it in a minute or two).

Do note that I have hours upon hours on the wget syntax, and I still can't
get this command to work, so I don't expect others to spend that kind of
time on it.

However, if there are wget experts out there, then I pose the question just
in case this finds you in the mood to help everyone obtain openvpn files
that are current (since they go stale within hours) from the openvpn.net
university free public vpn site.

I'll explain more details in a followup.
Merry Christmas!

Ads
  #2  
Old December 25th 16, 06:22 PM posted to alt.comp.os.windows-10,alt.windows7.general
Hazuki Nakamura
external usenet poster
 
Posts: 78
Default Wget question which only an expert can answer (obtain vpn config files from vpngate.net)

Hazuki Nakamura said:

The expert only question is how can we get this wget command to work?

c:\ wget --wait=3 --random-wait -r -I /en,/common -A
do_openvpn.aspx,openvpn_download.aspx\* -nc -nd http://www.vpngate.net/en/
-e robots=off


Here is the problem set, where I will download three files manually to show
you the manual process that would benefit all if we can get wget to work.

1. Point your browser to http://vpngate.net
2. Do a Control+F "find" for "config file"
3. Right click on that "config file" link and "open link in new tab"
4. Right click on the first OpenVPN file & "save link as" whatever.ovpn

Do that manually for as many config files as you like, e.g.,
"Save link as"
vpngate_vpn954935463.opengw.net_udp_1224.ovpn
vpngate_vpn330417614.opengw.net_udp_1195.ovpn
vpngate_vpn544885811.opengw.net_udp_1310.ovpn
vpngate_vpn489613170.opengw.net_udp_1195.ovpn
etc.

The only question here is how to save those files using wget, which only a
wget expert will be able to solve because the desired file is always on the
second "do_openvpn.aspx" page.
http://www.vpngate.net/en/do_openvpn...&hi d=6462250

Wget seems like the correct tool, but I have struggled for many hours, and
failed to get wget to download the files.
https://www.gnu.org/software/wget/manual/wget.html

Manually I can concatonate the URL plus the file name but I have to know
both ahead of time (i.e., URL&/filename.ovpn)

c:\ wget
http://www.vpngate.net/en/do_openvpn..._udp_1224.ovpn

The best I can do to automate that (which fails so far due to syntax) is:
wget
--wait=3 --random-wait (optional random wait to not hammer the server)
-r (recursive)
-I /en,/common (include a comma-separated directory list)
-A do_openvpn.aspx,openvpn_download.aspx\* (filename accept list)
-nc (noclobber, i.e., don't download the same file twice)
-nd (nodirectories, i.e., don't create a bunch of directories)
http://www.vpngate.net/en/ (this is the starting point URL)
-e robots=off (execute the command to ignore the server robot directive)

Is there a wget expert out there who can download ovpn files from
vpngate.net?
  #3  
Old December 25th 16, 07:16 PM posted to alt.comp.os.windows-10,alt.windows7.general
Paul[_32_]
external usenet poster
 
Posts: 11,873
Default Wget question which only an expert can answer (obtain vpn configfiles from vpngate.net)

Hazuki Nakamura wrote:
The expert only question is how can we get this wget command to work?

c:\ wget --wait=3 --random-wait -r -I /en,/common -A
do_openvpn.aspx,openvpn_download.aspx\* -nc -nd http://www.vpngate.net/en/
-e robots=off

I must warn any respondents that this wget syntax question is so difficult
to answer that I do not expect any answer (except perhaps from someone who
is so proficient in wget syntax that they can code it in a minute or two).

Do note that I have hours upon hours on the wget syntax, and I still can't
get this command to work, so I don't expect others to spend that kind of
time on it.

However, if there are wget experts out there, then I pose the question just
in case this finds you in the mood to help everyone obtain openvpn files
that are current (since they go stale within hours) from the openvpn.net
university free public vpn site.

I'll explain more details in a followup.
Merry Christmas!


You would start with actual working URLs, before crafting a WGET for them.

This is an example of the URL of a config file (.ovpn)

http://www.vpngate.net/common/openvp..._udp_1260.ovpn

You appear to have defined the top of your crawl as

http://www.vpngate.net/en/

yet the "goods" are in

http://www.vpngate.net/common

At key to your design, is what does "regex" mean in
the user manual ? Does it mean tight adherence to
actual regex (which is an unbounded way of specifying
thing) ? Is it actually an instance of "PCRE" (which
is bounded and relatively safe). When they write a manual
and give examples, I'm getting a bad smell from this.
The examples don't seem to be escaping character
sequences the way I would expect. If you stray from
"standards", your manual must provide copious examples
of acceptable syntax. I don't think GNU strayed, but
others who forked this may have.

https://www.gnu.org/software/wget/manual/wget.html

It would seem to me, the gnuwin32 version doesn't
have regex at all for the Accept list. But there are
other versions (ports) available.

I don't think you need a rocket scientist. But
you will have fun figuring out what "starting materials"
you've got, and what manual page actually corresponds
to how the EXE you've got, works. The wget 1.11.4 I have
sitting on disk here, doesn't accept --regex-type for
example.

I might craft the command like this, but I could be quite wrong.

wget.exe --wait=3 --random-wait -r -I /en,/common -A do_openvpn.aspx,openvpn_download.aspx
-nc -nd http://www.vpngate.net/ -e robots=off

What that does, is presumably matches on the entire
string (with comma separation).

Or, I might try it this way. The assumption being,
all the info is in the "common" directory, so why quibble.

wget.exe --wait=3 --random-wait -r -A openvpn_download.aspx
-nc -nd http://www.vpngate.net/common/ -e robots=off

But whether it's Regex, PCRE, or simple string matching,
that's going to make a difference as to the (filtered) results.

It's a statically compiled binary - running DependencyWalker
on it tells me nothing. And looking at a couple of
source tarballs tells me nothing either, because
the "port" might have removed the regex for example.

*******

Be very careful with wget. You can pull down way more
info than necessary, ****ing off the web site operator
and triggering "anti-hammering" code. This will cause the
web server to stop serving your IP address. Simply
drop the lease on your connection, so your ISP gives
a new dynamic IP address and try again. If a human is
present at the web site console, they could block your
ISP if it is determined you're trying to do a denial
of service attack. There are nation-states who have probably
tried to do that, to the server in question. In extreme
cases, the services of CloudFlare might be required.

Paul
  #4  
Old December 25th 16, 09:27 PM posted to alt.comp.os.windows-10,alt.windows7.general
JJ[_11_]
external usenet poster
 
Posts: 744
Default Wget question which only an expert can answer (obtain vpn config files from vpngate.net)

On Sun, 25 Dec 2016 16:41:29 +0000 (UTC), Hazuki Nakamura wrote:
The expert only question is how can we get this wget command to work?

c:\ wget --wait=3 --random-wait -r -I /en,/common -A
do_openvpn.aspx,openvpn_download.aspx\* -nc -nd http://www.vpngate.net/en/
-e robots=off

I must warn any respondents that this wget syntax question is so difficult
to answer that I do not expect any answer (except perhaps from someone who
is so proficient in wget syntax that they can code it in a minute or two).

Do note that I have hours upon hours on the wget syntax, and I still can't
get this command to work, so I don't expect others to spend that kind of
time on it.

However, if there are wget experts out there, then I pose the question just
in case this finds you in the mood to help everyone obtain openvpn files
that are current (since they go stale within hours) from the openvpn.net
university free public vpn site.

I'll explain more details in a followup.
Merry Christmas!


You'll need a scripting language for this due to these reasons:

- The config file download URL changes dynamically since it includes session
ID.

- It needs two steps (from the front page) in order to get the actual
download URL. The front page, and the config file download page.

- Parsing the page HTML code is needed in order to get each the config file
page URLs on the front page, and the config file download URLs on the config
file download page.

IMO, I'd use VBScript to fetch and parse the HTML pages, and use WGet to
download the config files in the background.
  #5  
Old December 25th 16, 11:21 PM posted to alt.comp.os.windows-10,alt.windows7.general
Paul[_32_]
external usenet poster
 
Posts: 11,873
Default Wget question which only an expert can answer (obtain vpn configfiles from vpngate.net)

JJ wrote:
On Sun, 25 Dec 2016 16:41:29 +0000 (UTC), Hazuki Nakamura wrote:
The expert only question is how can we get this wget command to work?

c:\ wget --wait=3 --random-wait -r -I /en,/common -A
do_openvpn.aspx,openvpn_download.aspx\* -nc -nd http://www.vpngate.net/en/
-e robots=off

I must warn any respondents that this wget syntax question is so difficult
to answer that I do not expect any answer (except perhaps from someone who
is so proficient in wget syntax that they can code it in a minute or two).

Do note that I have hours upon hours on the wget syntax, and I still can't
get this command to work, so I don't expect others to spend that kind of
time on it.

However, if there are wget experts out there, then I pose the question just
in case this finds you in the mood to help everyone obtain openvpn files
that are current (since they go stale within hours) from the openvpn.net
university free public vpn site.

I'll explain more details in a followup.
Merry Christmas!


You'll need a scripting language for this due to these reasons:

- The config file download URL changes dynamically since it includes session
ID.

- It needs two steps (from the front page) in order to get the actual
download URL. The front page, and the config file download page.

- Parsing the page HTML code is needed in order to get each the config file
page URLs on the front page, and the config file download URLs on the config
file download page.

IMO, I'd use VBScript to fetch and parse the HTML pages, and use WGet to
download the config files in the background.


By removing the Accept (-A) argument, I can get some results.

wget.exe --wait=3 --random-wait -r -I /en,/common -nc -nd http://www.vpngate.net/ -e robots=off

Now, it's a matter of figuring out where the element references
come from.

The -A accept doesn't seem to be working worth a damn
on my gnuwin32 copy of WGET. It was rejecting everything.
The documentation seems to suggest it is intended for
"jpg,gif" kind of syntax.

And I can't tell if wget still spiders properly if it
doesn't save any files. It doesn't seem to. If it gets
a chance to save index.html, it seems to soldier onwards.

The web pages also claim to only serve a percentage of
all servers at a time.

Paul
  #6  
Old December 26th 16, 12:55 AM posted to alt.comp.os.windows-10,alt.windows7.general
Mike Tomlinson
external usenet poster
 
Posts: 654
Default Wget question which only an expert can answer (obtain vpn config files from vpngate.net)

En el artículo , Paul
escribió:

wget.exe --wait=3 --random-wait -r -I /en,/common -nc -nd
http://www.vpngate.net/ -e robots=off


The -A accept doesn't seem to be working worth a damn
on my gnuwin32 copy of WGET. It was rejecting everything.


could it be getting confused by the \ ? Try quoting the arguments?

wget.exe --wait=3 --random-wait -I "/en","/common" -nc -nd http://foo.bar

--
(\_/)
(='.'=) systemd: the Linux version of Windows 10
(")_(")
  #7  
Old December 26th 16, 01:11 AM posted to alt.comp.os.windows-10,alt.windows7.general
Paul[_32_]
external usenet poster
 
Posts: 11,873
Default Wget question which only an expert can answer (obtain vpn configfiles from vpngate.net)

Mike Tomlinson wrote:
En el artículo , Paul
escribió:

wget.exe --wait=3 --random-wait -r -I /en,/common -nc -nd
http://www.vpngate.net/ -e robots=off


The -A accept doesn't seem to be working worth a damn
on my gnuwin32 copy of WGET. It was rejecting everything.


could it be getting confused by the \ ? Try quoting the arguments?

wget.exe --wait=3 --random-wait -I "/en","/common" -nc -nd http://foo.bar


That part is strangely working. It does confine the
crawl to /en and /common as requested.

I might be tempted to pull in the Cygwin version
rather than the GNUWIN32 version of wget and try again.
If there was a purpose.

The line above does pull in some materials. The OP
might have some clever way of sorting through them.

I agree with "jj" that two stages is probably
a better approach. Pull in the index.html file
first, extract the desired class of URLs, then
only download those. For example, a findstr
on "do_openvpn.aspx?fqdn" might be a start.

findstr do_openvpn.aspx?fqdn index.html

The above command is awfully slow, which is a good
reason for re-writing the approach somehow.

HTH,
Paul
  #8  
Old December 26th 16, 01:23 AM posted to alt.comp.os.windows-10,alt.windows7.general
Hazuki Nakamura
external usenet poster
 
Posts: 78
Default Wget question which only an expert can answer (obtain vpn config files from vpngate.net)

Paul said:

You appear to have defined the top of your crawl as
http://www.vpngate.net/en/
yet the "goods" are in
http://www.vpngate.net/common


Hello Paul,

Thanks for the additional advice on this very complex wget operation.
I knew the problem set would not be easy.

Yours is a healthy observation, which I hadn't realized, that I was
starting where I start manually, but then I was trying to get wget to grab
files which turned out to be elsewhere based on the link at the top-level
page.

It would seem to me, the gnuwin32 version doesn't
have regex at all for the Accept list. But there are
other versions (ports) available.


Oh oh. That might be the problem then!
The Windows version of wget may not handle regex the way the manual
implied.

I don't think you need a rocket scientist. But
you will have fun figuring out what "starting materials"
you've got, and what manual page actually corresponds
to how the EXE you've got, works. The wget 1.11.4 I have
sitting on disk here, doesn't accept --regex-type for
example.


I'm using the same version as you a
C:\Documents and Settings\Doc wget --version
----------- begin output
SYSTEM_WGETRC = c:/progra~1/wget/etc/wgetrc
syswgetrc = C:\apps\os\gnu_get/etc/wgetrc
GNU Wget 1.11.4

Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
http://www.gnu.org/licenses/gpl.html.
This is free softwa you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Originally written by Hrvoje Niksic .
Currently maintained by Micah Cowan .
----------- end output

I might craft the command like this, but I could be quite wrong.
wget.exe --wait=3 --random-wait -r -I /en,/common -A do_openvpn.aspx,openvpn_download.aspx
-nc -nd http://www.vpngate.net/ -e robots=off


Start Run cmd
mkdir \tmp\test1
cd \tmp\test1
wget.exe --wait=3 --random-wait -r -I /en,/common -A
do_openvpn.aspx,openvpn_download.aspx -nc -nd http://www.vpngate.net/ -e
robots=off

It seemed to work to completion, but after a long time the directory was
empty.

Or, I might try it this way. The assumption being,
all the info is in the "common" directory, so why quibble.

wget.exe --wait=3 --random-wait -r -A openvpn_download.aspx
-nc -nd http://www.vpngate.net/common/ -e robots=off


Start Run cmd
mkdir \tmp\test2
cd \tmp\test2
wget.exe --wait=3 --random-wait -r -A openvpn_download.aspx -nc -nd
http://www.vpngate.net/common/ -e robots=off

From my ISP I got a "forbidden" (maybe I ran too many tests?) but when I
went onto VPN, it seemed to chunk away:

wget.exe --wait=3 --random-wait -r -A openvpn_download.aspx -nc -nd
http://www.vpngate.net/common/ -e robots=off
---------- first response below from my isp
SYSTEM_WGETRC = c:/progra~1/wget/etc/wgetrc
syswgetrc = C:\apps\os\gnu_get/etc/wgetrc
--2016-12-25 15:23:09-- http://www.vpngate.net/common/
Resolving www.vpngate.net... 130.158.75.36, 130.158.75.44, 130.158.75.40,
....
Connecting to www.vpngate.net|130.158.75.36|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2016-12-25 15:23:09 ERROR 403: Forbidden.
---------- from that can I surmise my IP address is forbidden?

Going onto VPN and trying again, it seems to work better:

C:\tmp\test2 wget.exe --wait=3 --random-wait -r -I /en,/common -A
do_openvpn.asp
x,openvpn_download.aspx -nc -nd http://www.vpngate.net/ -e robots=off
----------- here is the response
SYSTEM_WGETRC = c:/progra~1/wget/etc/wgetrc
syswgetrc = C:\apps\os\gnu_get/etc/wgetrc
--2016-12-25 15:45:34-- http://www.vpngate.net/
Resolving www.vpngate.net... 130.158.75.36, 130.158.75.44, 130.158.75.33,
....
Connecting to www.vpngate.net|130.158.75.36|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: /en/ [following]
--2016-12-25 15:45:34-- http://www.vpngate.net/en/
Reusing existing connection to www.vpngate.net:80.
HTTP request sent, awaiting response... 200 OK
Length: 194145 (190K) [text/html]
Saving to: `index.html'

100%[======================================] 194,145 51.9K/s in 3.7s
etc.
  #9  
Old December 26th 16, 01:23 AM posted to alt.comp.os.windows-10,alt.windows7.general
Hazuki Nakamura
external usenet poster
 
Posts: 78
Default Wget question which only an expert can answer (obtain vpn config files from vpngate.net)

Paul said:

The web pages also claim to only serve a percentage of
all servers at a time.


I'm behind in the responses because each one is chock full of details to
try, but to just comment on the number of servers, that shouldn't matter
because the wget would get the *current* list of server files.

That current list changes (constantly) but it doesn't change so fast that a
single wget would fail.

It probably changes every day or every half day or every quarter day.
Certainly if I log in tomorrow, many will be the same and many will be new.

The old ones still work though, as some old files work for weeks and very
few for months, but most of the files only last a day or three.

That's why the wget is so important, because it would help others obtain
the vpn files, and not just me (since this is an open university site).
  #10  
Old December 26th 16, 01:37 AM posted to alt.comp.os.windows-10,alt.windows7.general
Paul[_32_]
external usenet poster
 
Posts: 11,873
Default Wget question which only an expert can answer (obtain vpn configfiles from vpngate.net)

Hazuki Nakamura wrote:
Paul said:

The web pages also claim to only serve a percentage of
all servers at a time.


I'm behind in the responses because each one is chock full of details to
try, but to just comment on the number of servers, that shouldn't matter
because the wget would get the *current* list of server files.

That current list changes (constantly) but it doesn't change so fast that a
single wget would fail.

It probably changes every day or every half day or every quarter day.
Certainly if I log in tomorrow, many will be the same and many will be new.

The old ones still work though, as some old files work for weeks and very
few for months, but most of the files only last a day or three.

That's why the wget is so important, because it would help others obtain
the vpn files, and not just me (since this is an open university site).


I just adjusted until I got some files out of the process.
Dropping the "-A" portion helped, because I couldn't get it
to work.

wget.exe --wait=3 --random-wait -r -I /en,/common -nc -nd http://www.vpngate.net/ -e robots=off

I created a folder first and started the process there, so
it wouldn't make a mess.

You might want to add a parameter to log all the URLs used
in that command, for later analysis (have wget dump its log
to a text file) . I didn't catch where the
index.html file came from, and it scrolled off my
Command Prompt window.

And what you're doing, is obtaining about 170 randomly
selected database entries out of 7700 of them. So rather
than being "old" and "new", they are only showing you
a subset of all entries available. They must have some
rules as to what portion of the database they "show" per day,
to prevent abuses (Denial Of Service against each VPN host).

There is also a notation that says some information is
randomly corrupted, for protection purposes.

I presume if you work at this long enough (a couple years),
you might eventually obtain all the available entries. But
without knowledge of how they keep track, it's hard to say.
They might try to use a client cookie, but a determined
attacker would just use a botnet to milk the thing.

Paul
  #11  
Old December 26th 16, 01:54 AM posted to alt.comp.os.windows-10,alt.windows7.general
Hazuki Nakamura
external usenet poster
 
Posts: 78
Default Wget question which only an expert can answer (obtain vpn config files from vpngate.net)

Mike Tomlinson said:

wget.exe --wait=3 --random-wait -I "/en","/common" -nc -nd http://foo.bar


Thank you for that help.
Using that syntax successfully downloads the index.html file.

C:\tmp\test2 wget.exe --wait=3 --random-wait -I "/en","/common" -nc -nd
http://www.vpngate.net/

Looking at that index.html file, I can see the URLs embedded inside by
searching for "do_openvpn.aspx"(or even just "do_") where a typical set of
links in that index.html file a

1. a href='
do_openvpn.aspx?fqdn=vpn621987741.opengw.net&ip=12 8.28.233.111&tcp=1372&udp=1373&sid=1482715846143&h id=565962
'

2. a href='
do_openvpn.aspx?fqdn=vpn952035314.opengw.net&ip=11 8.41.60.103&tcp=995&udp=1195&sid=1482715846143&hid =5627333
'

3. a href='
do_openvpn.aspx?fqdn=vpn345565696.opengw.net&ip=73 .25.142.235&tcp=1714&udp=1340&sid=1482715846143&hi d=4962280
'

I'm using VIM at the moment, on Windows, to see if I can figure out the
regular expressions that will pull those "do_" URLs out of the file.

This is intended to make each URL its own line in the file:
:%s/do_openvpn.*\'/\r&\r/g

:% (for the entire file)
/ (search for)
do_openvpn (the character do_openvpn)
..* (any character after that and any number of those characters)
\' (until you get to the *first* single quote) ==== this is not working
/ (replace what you found with)
\r (a line return at the beginning of what you found)
& (then put what you found, which is the do_openvpn line up to the quote)
\r (then put another line return at the end of what you found)
/g (and do that globally throughout the file)

At the moment, the only problem that VIM syntax gives me is that it finds
the *last* single quote of the line, and not the first!

Once I get the do_openvpn line all by itself, I can easily sort or grep for
just the do_openvpn lines, so I just have to figure out why my VIM regular
expressions above are catching the *last* single quote in the line instead
of the first one after the do_openvpn characters.
  #12  
Old December 26th 16, 02:08 AM posted to alt.comp.os.windows-10,alt.windows7.general
Hazuki Nakamura
external usenet poster
 
Posts: 78
Default Wget question which only an expert can answer (obtain vpn config files from vpngate.net)

Hazuki Nakamura said:

Once I get the do_openvpn line all by itself, I can easily sort or grep for
just the do_openvpn lines, so I just have to figure out why my VIM regular
expressions above are catching the *last* single quote in the line instead
of the first one after the do_openvpn characters.


The single quote must be an odd character because I can get the regular
expression to work inside of VIM if I use "img" instead of the single
quote.

vim index.html
:%s/do_.*img/\r&\r/g
:!sort

That puts all the do_openvpn lines together where I only need to strip out
the trailing 'img (which was why I was trying to use the quote as the
ending character instead of the letters "img").

Note that a grep would be faster than a sort but I don't have cygwin
running yet.

INTENT:
:%s/do_.*img/\r&\r/g

: === execute a command in VIM
% === over the entire file
s/ === search for
do_ === the characters "do_"
..* === any character after that and any number of those
img === until you see the lower-case letters "img"
/ === replace that with
\r === carriage return (so that its on its own line)
& === what you found between the do_ and the "img" characters
\r === a carriage return at the end of that
/g === and do that globally for every time you find it

The sort is designed to put all the do_openvpn lines together, but it's
taking forever so I think I need to kill any line that doesn't start with
do_openvpn instead.
  #13  
Old December 26th 16, 02:20 AM posted to alt.comp.os.windows-10,alt.windows7.general
Hazuki Nakamura
external usenet poster
 
Posts: 78
Default Wget question which only an expert can answer (obtain vpn config files from vpngate.net)

Hazuki Nakamura said:

The sort is designed to put all the do_openvpn lines together, but it's
taking forever so I think I need to kill any line that doesn't start with
do_openvpn instead.


This worked to delete all the lines that did *not* start with "do_".
:%s/^[^do_openvpn].*//g

That resulted in 411 lines of the form:
do_openvpn.aspx?fqdn=vpn201597492.opengw.net&ip=20 2.163.188.135&tcp=1421&udp=1449&sid=1482715846159& hid=4136512'img
do_openvpn.aspx?fqdn=vpn865252102.opengw.net&ip=18 0.64.47.18&tcp=1760&udp=1451&sid=1482715846159&hid =1440591'img
do_openvpn.aspx?fqdn=vpn478313674.opengw.net&ip=21 8.127.153.160&tcp=1690&udp=1967&sid=1482715846159& hid=433070'img

INTERPRETATION:
:% === run the command on the entire file
s/ === search for
^ === the beginning of the line
[^do_openvpn].* === NOT beginning with do_openvpn, plus any characters
/ === replace what you found with
null === nothing (i.e., delete all lines NOT starting with "do_")
/g === and do that repeatedly as needed

I don't know why this takes forever (and fails in the end):
:!sort

But this works quickly:
:'a,'b!sort

Where I marked a at the first line of the file by typing "ma" and I marked
b at the end of the file by typing "mb".

The only cleanup I needed was to delete the blank lines en masse and to get
rid of the 'img extraneous characters.

:%s/'img//g

INTERPRETATION:
:% === run the command on the entire file
s/ === search for
'img == the characters "'img"
/ === replace what you found with
null === nothing (i.e., delete all lines NOT starting with "do_")
/g === and do that repeatedly as needed

Here is a snipped resulting set of about 100 lines that resulted:
do_openvpn.aspx?fqdn=jrcollado1987.opengw.net&ip=1 62.219.5.10&tcp=443&udp=1194&sid=1482715846159&hid =5923234
do_openvpn.aspx?fqdn=rumow.opengw.net&ip=194.67.21 3.44&tcp=995&udp=1195&sid=1482715846159&hid=646774 8
do_openvpn.aspx?fqdn=vpn115521379.opengw.net&ip=12 5.161.103.254&tcp=0&udp=15369&sid=1482715846159&hi d=5249978
do_openvpn.aspx?fqdn=vpn135567190.opengw.net&ip=10 6.246.170.66&tcp=1795&udp=1349&sid=1482715846159&h id=4464380
do_openvpn.aspx?fqdn=vpn139802159.opengw.net&ip=20 1.249.118.70&tcp=1195&udp=1195&sid=1482715846143&h id=6302468
do_openvpn.aspx?fqdn=vpn150198121.opengw.net&ip=42 .127.69.52&tcp=1242&udp=1236&sid=1482715846159&hid =1111507
do_openvpn.aspx?fqdn=vpn179147953.opengw.net&ip=95 .18.124.225&tcp=1791&udp=1418&sid=1482715846159&hi d=4206253
do_openvpn.aspx?fqdn=vpn183301952.opengw.net&ip=66 .181.186.36&tcp=0&udp=18240&sid=1482715846159&hid= 5964917
do_openvpn.aspx?fqdn=vpn190967718.opengw.net&ip=70 .124.158.72&tcp=1988&udp=1565&sid=1482715846159&hi d=5356985
do_openvpn.aspx?fqdn=vpn191176579.opengw.net&ip=58 .124.236.216&tcp=1904&udp=1982&sid=1482715846159&h id=2538672
do_openvpn.aspx?fqdn=vpn198097305.opengw.net&ip=17 6.115.137.199&tcp=0&udp=4289&sid=1482715846159&hid =6251213
do_openvpn.aspx?fqdn=vpn201486029.opengw.net&ip=19 0.77.224.233&tcp=0&udp=29598&sid=1482715846159&hid =6230363
do_openvpn.aspx?fqdn=vpn201542698.opengw.net&ip=11 3.22.181.10&tcp=1341&udp=1530&sid=1482715846143&hi d=6045917
etc.

I can easily put a wget in front of that using VIM and then that's a
script, right?
  #14  
Old December 26th 16, 02:27 AM posted to alt.comp.os.windows-10,alt.windows7.general
Hazuki Nakamura
external usenet poster
 
Posts: 78
Default Wget question which only an expert can answer (obtain vpn config files from vpngate.net)

Hazuki Nakamura said:

I can easily put a wget in front of that using VIM and then that's a
script, right?


I guess the batch process is to put wget and doublequotes on each line:

:%s/^do_.*$/wget.exe "&"/g

INTERPRETATION:
:%s/ === search the entire file for
^do_.*$ === lines that start with do_ and select to the end of the line
/ === replace what you found with
wget.exe " === the characters wget.exe "
& === the line you found
" === the character "
/g === and repeat that as needed

That results in a batch file of the form:

wget.exe
"do_openvpn.aspx?fqdn=jrcollado1987.opengw.net&ip= 162.219.5.10&tcp=443&udp=1194&sid=1482715846159&hi d=5923234"
wget.exe
"do_openvpn.aspx?fqdn=rumow.opengw.net&ip=194.67.2 13.44&tcp=995&udp=1195&sid=1482715846159&hid=64677 48"
wget.exe
"do_openvpn.aspx?fqdn=vpn115521379.opengw.net&ip=1 25.161.103.254&tcp=0&udp=15369&sid=1482715846159&h id=5249978"
  #15  
Old December 26th 16, 02:29 AM posted to alt.comp.os.windows-10,alt.windows7.general
Hazuki Nakamura
external usenet poster
 
Posts: 78
Default Wget question which only an expert can answer (obtain vpn config files from vpngate.net)

Hazuki Nakamura said:

That results in a batch file of the form:

wget.exe
"do_openvpn.aspx?fqdn=jrcollado1987.opengw.net&ip= 162.219.5.10&tcp=443&udp=1194&sid=1482715846159&hi d=5923234"
wget.exe
"do_openvpn.aspx?fqdn=rumow.opengw.net&ip=194.67.2 13.44&tcp=995&udp=1195&sid=1482715846159&hid=64677 48"
wget.exe
"do_openvpn.aspx?fqdn=vpn115521379.opengw.net&ip=1 25.161.103.254&tcp=0&udp=15369&sid=1482715846159&h id=5249978"


Darn. That didn't download anything.
I realize belatedly, I need the ending to be *.ovpn but the process should
have worked. I goofed somewhere so I will back up.
 




Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off






All times are GMT +1. The time now is 06:08 PM.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 PCbanter.
The comments are property of their posters.