Apache Server configuration support - URL rewriting and proxying

Author

URL rewriting and proxying

Bob

2006-05-12, 7:16 pm

Now that I have my proxy working, I need to know how to perform URL
rewrites *before* passing along to the proxy.

e.g. I currently have this proxy working:

http://dev.igods.com --> http://www.pacificare.com

The first rewrite I need to manage is to get around the home page
redirect. if you go to:

http://www.pacificare.com/

you get redirected to:

http://www.pacificare.com/commonportal/index.jsp

I need to write 2 rewrite rules (that take effect *before* getting
proxied to the target domain) that match / and /index.jsp and proxies
(not redirects) to /commonportal/index.jsp

(I guess I could do it with one, but why cause a regexp migrane?)

Here are my attempted rewrite rules, and the accompanying proxy, all in
a nice virtualhost block (no snipping ;o)

<VirtualHost *:80>
ServerName dev.igods.com
ServerAdmin admin@igods.com
CustomLog "/var/log/httpd/access_log" "%{PC-Remote-Addr}i %l %u
%t \"%r\" %>s %b"
ErrorLog "/var/log/httpd/error_log"
ErrorDocument 404 /error.html
<IfModule mod_rewrite.c>

RewriteEngine On
RewriteLog /var/log/httpd/rewrite_log
RewriteLoglevel 5

# home page rewrite (root page version)
# incoming: http://www.pacificare.com/index.jsp
# target:
http://www.pacificare.com/commonPortal/index.jsp
RewriteCond %{REQUEST_URI}
/index.jsp
RewriteRule ^([^/]*)/index.jsp$
$1/commonportal/index.jsp

# incoming: http://www.pacificare.com/
# target:
http://www.pacificare.com/commonPortal/index.jsp
RewriteCond %{REQUEST_URI} /$
RewriteRule ^([^/]*)/$
$1/commonportal/index.jsp

</IfModule>
<IfModule mod_proxy.c>
ProxyRequests Off
ProxyPass / http://www.pacificare.com/
ProxyPassReverse / http://www.pacificare.com/
</IfModule>
<IfModule mod_alias.c>
</IfModule>
LogLevel warn
</VirtualHost>

However, when I try to go to http://dev.igods.com/index.jsp , I get the error:

Not Found

The requested URL /index.jsp was not found on this server.
Apache/2.0.54 (Unix) mod_jk/1.2.6 mod_ssl/2.0.54 OpenSSL/0.9.7c DAV/2
Server at www.pacificare.com Port 80

.... which tells me that it's not getting rewritten before the proxy,
since dev.igods.com is running Apache 1.3, while 2.054 is at the client
site. (and indeed, http://www.pacificare.com/index.jsp doesn't exist.)

Unfortunately, this is the order I need to pull off. The logic of the
underlying web application at the site is such that if I try to rewrite
URLs at the site itself, it 403 blocks me... undoubtedly, its running
some verifications against the url and not liking the rewrites it finds.

So, my idea was to proxy the site, and do the rewriting BEFORE it even
lands at the web application.

Yes, I want my cake, and to eat it too.

Ideas? Am I off base here, is the rewrite engine not even turned on? I
did check httpd.conf, and verified:

LoadModule rewrite_module libexec/httpd/mod_rewrite.so
AddModule mod_rewrite.c
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_METHOD} ^TRACE
RewriteRule .* - [F]
</IfModule>

thanks folks, you've been a charm!
b.

Robert Ionescu

2006-05-14, 7:14 pm

Bob wrote:
> I need to write 2 rewrite rules (that take effect *before* getting
> proxied to the target domain) that match / and /index.jsp and proxies
> (not redirects) to /commonportal/index.jsp

Can't you use mod_rewrite with the P flag? That should pass teh result
along to mod_proxy /ProxyPass

RewriteEngine On
RewriteLog /var/log/httpd/rewrite_log
RewriteLoglevel 5

RewriteRule ^/(index\.jsp)?$ /commonportal/index.jsp
RewriteRule ^/(.*) http://www.pacificare.com/$1 [P]
ProxyPassReverse / http://www.pacificare.com/

--
Robert

Bob

2006-05-16, 1:17 pm

On 2006-05-14 13:24:56 -0700, Robert Ionescu <robsiegen@googlemail.com> said:

> Bob wrote:
>
> Can't you use mod_rewrite with the P flag? That should pass teh result
> along to mod_proxy /ProxyPass
>
> RewriteEngine On
> RewriteLog /var/log/httpd/rewrite_log
> RewriteLoglevel 5
>
> RewriteRule ^/(index\.jsp)?$ /commonportal/index.jsp
> RewriteRule ^/(.*) http://www.pacificare.com/$1 [P]
> ProxyPassReverse / http://www.pacificare.com/

Thanks Robert, that did the trick... I have my first rewritten URL
working (a much more complex one than the example above, but the same
principle applies).

I'm no longer using the proxy rules, so I'm rewriting the URLs within
the same domain (i.e. pacificare.com --> pacificare.com).

Now, given this example:

www.pacificare.com/test.html --> www.pacificare.com/index.html?test

(where the ?test URL is the final URL, and test.html is the 'pretty'
URL ), I'm now able to surf to the pretty URL, and the pretty URL
doesn't change in the browser's location bar. The last step in the
process is to have the site itself expose the pretty URLs, rather than
the dynamic URLs. (This is the reason I thought to use a proxy.)

In other words, keeping to the same example, I need the site to expose
the /test.html, rather than the index.html?test

Is this possible without using the proxy?

thanks!
b.

Robert Ionescu

2006-05-17, 7:18 am

Bob wrote:
> I need the site to expose
> the /test.html, rather than the index.html?test

It's just a normal rewrite like

# not matching subdirectories
RewriteRule ^/([^/]+)\.html$ /index.php?$1 [L]

Bob

2006-05-17, 1:17 pm

On 2006-05-17 01:02:34 -0700, Robert Ionescu <robsiegen@googlemail.com> said:

> Bob wrote:
>
> It's just a normal rewrite like
>
> # not matching subdirectories
> RewriteRule ^/([^/]+)\.html$ /index.php?$1 [L]

I'm not sure I conveyed my intent. Currently, there exists on the site
urls of the form:

www.example.com/someapp?pageid=test

I've successfully written rewrite rules that allow me to surf to:

www.example.com/test.html

and display the someapp?pageid=test correctly. However, I still see
(and the search engines still see) the someapp?pageid=test url.

I need instead the site to expose the rewriten urls to the world, such
that if you surf to the site and view source, you'll see the test.html
url, *not* the someapp?pageid=test url.

the normal rewrites allow me to surf *to* the pretty urls, but don't
expose those pretty urls to the world, which is what I'm after.

ideas?

thanks much, couldn't do this without out you. (yes, I've read apache's
docs on the rewrites, but they haven't clarified this for me.)

cheers
b.

Jim Hayter

2006-05-17, 7:17 pm

Bob wrote:
> On 2006-05-17 01:02:34 -0700, Robert Ionescu <robsiegen@googlemail.com>
> said:
>
>
>
> I'm not sure I conveyed my intent. Currently, there exists on the site
> urls of the form:
>
> www.example.com/someapp?pageid=test
>
>
> I've successfully written rewrite rules that allow me to surf to:
>
> www.example.com/test.html
>
> and display the someapp?pageid=test correctly. However, I still see (and
> the search engines still see) the someapp?pageid=test url.
>
> I need instead the site to expose the rewriten urls to the world, such
> that if you surf to the site and view source, you'll see the test.html
> url, *not* the someapp?pageid=test url.
>
> the normal rewrites allow me to surf *to* the pretty urls, but don't
> expose those pretty urls to the world, which is what I'm after.
>
> ideas?
>
> thanks much, couldn't do this without out you. (yes, I've read apache's
> docs on the rewrites, but they haven't clarified this for me.)
>
> cheers
> b.
>

You just need a rule that detects the someapp?pageid=test url. You can
use RewriteCond to parse the query_string into variables, then a rewrite
rule to formulate the pretty url. Add the flag to return a permanent
redirect status. Note that %1 matches the parenthesized expression in
the RewriteCond.

* untested *
RewriteCond %{QUERY_STRING} ^pageid=(.*)$
RewriteRule ^/someapp$ /%1.html [R=permanent,L]

HTH,
Jim

Bob

2006-05-18, 1:22 am

On 2006-05-17 14:19:42 -0700, Jim Hayter <see.reply.to@nowhere.invalid> said:
>
> You just need a rule that detects the someapp?pageid=test url. You can
> use RewriteCond to parse the query_string into variables, then a
> rewrite rule to formulate the pretty url. Add the flag to return a
> permanent redirect status. Note that %1 matches the parenthesized
> expression in the RewriteCond.
>
> * untested *
> RewriteCond %{QUERY_STRING} ^pageid=(.*)$
> RewriteRule ^/someapp$ /%1.html [R=permanent,L]
>
> HTH,
> Jim

Thanks for the help Jim, I understand a little better... especially
using QUERY_STRING rather than REQUEST_URI for matching. (although, I'm
now not sure how I'd match on a combination of the 2... can I use two
RewriteCond lines in a row to make an AND test?)

Unfortunately, this didn't seem to do the trick. The URL:

http://www.pacificare.com/commonPor...fromGateway=yes

still

appears identically as:

http://www.pacificare.com/commonPor...fromGateway=yes

when

surfing to the site. I need the URL to look like this when I surf to the site:

http://www.pacificare.com/navnode/G...fromGateway/yes

This is the rule I tried:

# navnode outward facing rewrite
RewriteCond %{QUERY_STRING} ^navnode=(.*)&fromGateway=(.*)$
RewriteRule ^/commonPortal/link$ /navnode/%1/fromGateway/%2 [R]

(I'm actually using a development environment, so you won't see any
evidence of this on the live site.)

Am I close, or out to lunch?

cheers,
b

Jim Hayter

2006-05-18, 1:17 pm

Bob wrote:
>
> Thanks for the help Jim, I understand a little better... especially
> using QUERY_STRING rather than REQUEST_URI for matching. (although, I'm
> now not sure how I'd match on a combination of the 2... can I use two
> RewriteCond lines in a row to make an AND test?)
>
Yes, two in a row default to an "AND". There is an "OR" flag to use if
you want that.

> Unfortunately, this didn't seem to do the trick. The URL:
>
> http://www.pacificare.com/commonPor...fromGateway=yes
>
>
> still
> appears identically as:
>
> http://www.pacificare.com/commonPor...fromGateway=yes
>
>
> when
> surfing to the site. I need the URL to look like this when I surf to the
> site:
>
>
> http://www.pacificare.com/navnode/G...fromGateway/yes
>
> This is the rule I tried:
>
> # navnode outward facing rewrite
> RewriteCond %{QUERY_STRING} ^navnode=(.*)&fromGateway=(.*)$
> RewriteRule ^/commonPortal/link$ /navnode/%1/fromGateway/%2 [R]

I'm not sure this will help. I'd try:
RewriteCond %{QUERY_STRING} ^navnode=([^&])&fromGateway=(.*)$

Another possibility is the R flag. Note that I used "[R=permanent,L]"
while you used "[R]". The R flag by itself is a temporary redirection.
This may be the browser not updating its displayed URL because the
change is temporary.

Let me know which, if either, of the above helps.

>
> (I'm actually using a development environment, so you won't see any
> evidence of this on the live site.)
>
> Am I close, or out to lunch?
>
> cheers,
> b

Good luck,
Jim

Bob

2006-05-18, 7:16 pm

On 2006-05-18 10:21:16 -0700, Jim Hayter <see.reply.to@nowhere.invalid> said:

> I'm not sure this will help. I'd try:
> RewriteCond %{QUERY_STRING} ^navnode=([^&])&fromGateway=(.*)$
>
> Another possibility is the R flag. Note that I used "[R=permanent,L]"
> while you used "[R]". The R flag by itself is a temporary redirection.
> This may be the browser not updating its displayed URL because the
> change is temporary.
>
> Let me know which, if either, of the above helps.
>
> Good luck,
> Jim

Still no luck. I tried this rule:

# navnode outward facing rewrite
RewriteCond %{REQUEST_URI} ^/commonPortal/link
RewriteCond %{QUERY_STRING} ^navnode=([^&]*)&fromGateway=([^&]*)$
RewriteRule ^/commonPortal/link.*$ /navnode/%1/fromGateway/%2
[R=permanent,L]

The rule did have some effect, in that it broke the URL:

http://wcq2.pacificare.com/commonPo...fromGateway=yes

(which

is still showing up on the home page as dynamic).

Removing the rule above makes the URL work again. Breaking the URL isn't
so bad, as it shows me that at least I am affecting the site, just not
correctly... good to know that I'm not affecting one thing and testing
another.

Robert, can you see where I'm going wrong here? I still don't understand
how these rewrite rules are supposed to affect the URLs on the site; they
seem designed to only affect incoming URLs (and I've had some success in
that regard), rather than affecting outward facing URLs.

thanks,
b.

Mike Gilbert

2006-05-20, 1:23 am

"Bob" <bob@nowhere.com> wrote in message
news:2006051812060816807-bob@nowherecom...
>
> Robert, can you see where I'm going wrong here? I still don't understand
> how these rewrite rules are supposed to affect the URLs on the site; they
> seem designed to only affect incoming URLs (and I've had some success in
> that regard), rather than affecting outward facing URLs.
>
> thanks,
> b.

Thats like asking why a lawnmower doesn't fly even though there is a
propeller in in.

Bob

2006-05-21, 1:19 am

On 2006-05-19 21:15:30 -0700, "Mike Gilbert" <mike.gilbert@hotmail.com> said:

>
> "Bob" <bob@nowhere.com> wrote in message
> news:2006051812060816807-bob@nowherecom...
>
>
> Thats like asking why a lawnmower doesn't fly even though there is a
> propeller in in.

perhaps you might actually write something useful, instead of showing
the group what an idiot you are? that might be something to keep to
yourself.

Mike Gilbert

2006-05-21, 1:19 am

"Bob" <bob@nowhere.com> wrote in message
news:2006052011480616807-bob@nowherecom...

> perhaps you might actually write something useful, instead of showing the
> group what an idiot you are? that might be something to keep to yourself.
>

My appologies Bob, I reeeally have to stay off the computer when I've been
drinking! I thought it was kind of funny at the time though. All I really
intended to point out is that while the links on the pages served by apache
may be urls, that does not necessitate their involvement with rewrite
rulesets.

Still, if you were under the impression that you could rewrite the urls on
pages served by apache through the use of mod rewrite, then probably the
most useful comment I could make would be this; dont mess with apache config
files until you have a better idea what you are doing! There be dragons
here!