UTF-8 URIs, mod_rewrite, and accents

6
Nov/08
2

Special characters (i.e. international characters with accents) weren’t showing up on our site, so I had to do this to fix it:

  1. Replace this meta tag: meta-http-equiv=”Content-Type” content=”txt/html; charset=iso-8859-1″
    With this meta tag: meta-http-equiv=”Content-Type” content=”txt/html; charset=utf-8″
  2. Add this to index.php
    header(’Content-Type: text/html; charset=utf-8′);
  3. Add this to .htaccess
    AddDefaultCharset UTF-8
    AddCharset UTF-8 .tpl
    AddCharset UTF-8 .js
    AddCharset UTF-8 .css
    AddCharset UTF-8 .php
  4. Change the database’s encoding from latin1 to UTF-8
  5. Change various tables’ encoding from latin1 to UTF-8
  6. Modify dbConnect.class.php so that it automatically runs these two queries EVERY TIME it creates a connection:
    1. SET CHARACTER SET utf8;
    2. SET NAMES utf8 COLLATE ‘utf8_general_ci’;
  7. Run a 1-time query to reset any incorrect database encodings:
    SET NAMES ‘utf8′ COLLATE ‘utf8_general_ci’;
    SET character_set_client = ‘utf8′;
    SET character_set_connection = ‘utf8′;
    SET character_set_database = ‘utf8′;
    SET character_set_results = ‘utf8′;
    SET character_set_server = ‘utf8′;
    SET collation_connection = ‘utf8_general_ci’;
    SET collation_database = ‘utf8_general_ci’;
    SET collation_server = ‘utf8_general_ci’;
    To double-check that this worked correctly, run this query: SHOW VARIABLES LIKE ‘c%’;
  8. Modify the MySQL config file:
    default-character-set=utf8
    default-collation=utf8_general_ci
    init-connect=’SET NAMES utf8′
    character_set_server=utf8
    character_set_client=utf8
    collation_server=utf8_general_ci

I thought that would fix everything, but it didn’t. Some of our URLs started causing Apache to explode, with an unexpected 404 “NOT FOUND” error. This link hated me: http://www.wikiDOMO.com/toronto_on/results/Caf√©

I Googled around for a good 4 hours, trying to find something about mod_rewrite, UTF-8, accented character URIs, internationalization, etc. I found lots, but nothing helped. I even enlisted the help of Chris Hartjes, Julian Simpson, and Jeff Kolesnikowicz, but we all came up empty-handed… so I went back to basics. Modifying all the RewriteRules one by one (we have 129 lines of them). Eventually I figured it out.

UTF-8 characters are not part of the a-zA-Z character set, so many of our re-write rules now failed.

To fix it, I simply had to change them from this ([a-zA-Z0-9_-])
To this (.
*)

Period means “any character”, and * means as many times as you like.

A few key articles:

Comments (2) Trackbacks (0)
  1. Juan
    1:28 pm on November 24th, 2008

    you’re in for a lot of surprises if you continue playing with UTF-8…. watch out for most string functions, usually they’ll have a mb_* equivalent.

    I’ve been “enjoying” utf-8 programming since I joined this company.

  2. Lex
    3:02 pm on April 12th, 2009

    You just saved me your 4 hours looking for a solution. Thanks a lot!

Leave a comment

No trackbacks yet.

PHP & AJAX

18
Aug/05
8

WOOT! This is entry #700 in my blog!
Given that I don’t post too too often, that’s pretty amazing :)
And now, on with the show…

Sorry Mom, but this post is mainly for php/web developers.

Today I’m going to mention some of the best stuff I’ve found lately.

Venkman is a great/complex Javascript debugger with breakpoints and stepping etc. Built by the Mozilla Foundation. Yes, it’s named after the Ghostbuster “Peter Venkman”, as played by Bill Murray.


Frustrated that there’s no Safari Javascript Console like the one Firefox has? Guess what! There is one! It’s just not easy to find, or turn on. Here’s how to do it:

  1. Type this line into terminal: defaults write com.apple.safari IncludeDebugMenu 1
  2. Start Safari
  3. In your Finder, to into the Applications=>Utilities Folder
  4. Drag the one named ‘Console’ to your Launch Bar
  5. Click it, and Voila, you have a Javascript Debug Console

We develop primarily for IE6 (ack!) at work, but since I’ve started implementing AJAX stuff they have asked me to make everything compatible with IE6 Windows, Firefox Windows/OSX, and Safari OSX. Fun fun fun!


Ever wanted an EASY way to validate form input?
Have it now. Have it easily. Check out The Form Assembly. Their killer “wForms” is an unobtrusive javascript extension that adds commonly needed behaviors to traditional web forms. It is cross-browser compatible, and does not require any knowledge of javascript to be used.


Follow the following link to know more about unobtrusive javascript


Tired of Internet Explorer Puking on CSS?
Dean Edwards has created an amazing javascript that you simply include in the head of your page, and it helps IE along whenever it feels a bit geriatric. He’s calling his script “IE7″. IE7 version 0.8 is now available for download.


As I’ve been doing a bunch of AJAX (asynchronous javascript and xml) lately, I have found some great AJAX resources/libraries. Here they are:

  • Prototype: The premier AJAX library, as used by Rico, RubyOnRails, Scriptaculous, and many more. No documentation, but otherwise very sweet.
  • script.aculo.us: A javascript class which extends Prototype, giving you more features. Amazing & easy to use. Slightly more documentation than Prototype.
  • Rico: Mind-bogglingly awesome javascript library that will change everything you think about javascript.
  • SAJAX: The simplest PHP/AJAX library I’ve found. It’s what I’m using at work.

Check out this cool Liquid Lens that will allow glass-lens quality optical zooming WITHOUT needing to extend or retract anything mechanically! It works like the human eye, or a glass of water that happens to magnify the newspaper sitting beneath it. Coming soon to cameraphones near you!


How many times have you needed a good, modifiable, easy-to-implement wysiwyg textarea editor? If you’re anything like me, the answer is many, and the number of times you’ve found one is zero, especially if your emphasis was ease of modifiability. Hunt no more, there’s now an open-source FREE alternative named TinyMCE and it looks pretty great… and FREE!


Are you a php developer? Do you want to be up-to-date on all the cool stuff going on in the php world? Do you want to get Zend Certified? If so, and if you have a lot of extra money laying around, you probably want to check out php|works. It’s a php conference happening in Toronto in mid-September. Registrants can write the Zend PHP Certification exam for free ($200 value), which is probably why it costs nearly $800 to register.

Many of the bigger names in PHP will be there, including:
Rasmus Lerdorf (the creator of php)
George Schlossnagle (guru – creator of APD/PDO/Serendipity)
Chris Shiflett (php security guru)
John Coggeshall (guru & Zend Technologies employee/consultant)
Wez Furlong (guru – prolific PEAR/PECL contributor & creator of PDO)
Too bad Harry Fuecks isn’t on the list. Oh well. Can’t win’em all.

I’m excited about going!!!
What should I wear???


Last, but certainly not least, a friend of mine recently built a really cool job search engine and named it JobBlender.com. What it does it aggregate listings from Workopolis, Monster.ca, and HotJobs, and give you search results from all 3 on a single easy to use page. You should try it :)

Comments (8) Trackbacks (0)
  1. mom
    11:15 pm on August 18th, 2005

    It doesn’t matter what its about, i love hearing about what you are interested in! I think Harry Fueks was on tv a few weeks ago, but have to be honest I wasn’t paying much attention. I think if you get a few hours you could design a very cool shirt. White dress, elegant, then add whatever php jargon you wish in a type design that looks like a tuxedo front. Well perhaps i am a bit odd, it was just a thought, I am so glad to hear that you are liking what you are doing at work, that is , to coin a phrase ” sweet ” hugs

  2. U.C.
    12:41 pm on August 20th, 2005

    Voodoo ether:well send your blog into space so E.T. can read your posts,check out http://bloginspace.com.

  3. Vlad
    6:51 pm on August 21st, 2005

    good idea with the jobblender.com

    a much better one is at http://www.petersnewjobs.com

  4. James Trott (UK)
    5:34 am on August 22nd, 2005

    Is job blender open source? i’d like to do something like that in the UK.

    - J

  5. tania
    8:55 pm on August 22nd, 2005

    “I’m excited about going!!!
    What should I wear???”

    uh… who are you and what have you done to derek? :)

  6. JB
    11:25 pm on August 22nd, 2005
  7. James Trott (UK)
    4:37 am on August 23rd, 2005

    So JB, are you actually going to write that? or are you just foolin, if so, then i could really use it right now for all the irish job sites (Job Blender.ie) as im currently looking for a job in ireland. Well, you seem so keen to blend jobs, i dont wanna steal your thunder :)

  8. Vlad
    10:30 am on August 23rd, 2005

    Derek is still Derek.. he needs us to help him choose between his Spiderman and Darth Vader costumes :)

Leave a comment

No trackbacks yet.

Comments (8) Trackbacks (0)
  1. mom
    11:15 pm on August 18th, 2005

    It doesn’t matter what its about, i love hearing about what you are interested in! I think Harry Fueks was on tv a few weeks ago, but have to be honest I wasn’t paying much attention. I think if you get a few hours you could design a very cool shirt. White dress, elegant, then add whatever php jargon you wish in a type design that looks like a tuxedo front. Well perhaps i am a bit odd, it was just a thought, I am so glad to hear that you are liking what you are doing at work, that is , to coin a phrase ” sweet ” hugs

  2. U.C.
    12:41 pm on August 20th, 2005

    Voodoo ether:well send your blog into space so E.T. can read your posts,check out http://bloginspace.com.

  3. Vlad
    6:51 pm on August 21st, 2005

    good idea with the jobblender.com

    a much better one is at http://www.petersnewjobs.com

  4. James Trott (UK)
    5:34 am on August 22nd, 2005

    Is job blender open source? i’d like to do something like that in the UK.

    - J

  5. tania
    8:55 pm on August 22nd, 2005

    “I’m excited about going!!!
    What should I wear???”

    uh… who are you and what have you done to derek? :)

  6. JB
    11:25 pm on August 22nd, 2005
  7. James Trott (UK)
    4:37 am on August 23rd, 2005

    So JB, are you actually going to write that? or are you just foolin, if so, then i could really use it right now for all the irish job sites (Job Blender.ie) as im currently looking for a job in ireland. Well, you seem so keen to blend jobs, i dont wanna steal your thunder :)

  8. Vlad
    10:30 am on August 23rd, 2005

    Derek is still Derek.. he needs us to help him choose between his Spiderman and Darth Vader costumes :)

Leave a comment

No trackbacks yet.

In other news…

4
Dec/03
3

If you rent a lot of movies and you somehow don’t yet own a dvd player, you will want to check this out. You rent the movies online, and they send them to you with a return postage envelope – so shipping is free. You get as many movies as you want each month for 25 bucks Canadian (i think you can only have X number checked out at once). ALSO, if you sign up now for a 2 year membership (at 25/month), you get a FREE KOSS DVD player. There’s never been a better time to spend money you don’t have!

I was working on a PHP Calendar application the other day, and discovered that I was born on a Tuesday, way back on April 26th, 1977.

Last night Tanya and I went out for a beer, which was quite nice. We had both been at the office a little late, so it was good to get out and relax (I was sending out the latest newsletter for TheLivingCentre.com, which, if you’re interested in hippy spiritual stuff at all, is a very cool place just outside of London, Ontario.

In other news, I should find out tomorrow or Monday if a web application I wrote for the Peterborough Health Unit will be rolled out province-wide. That would be cool. I did the programming as an employee of Solid Computing Solutions out of Peterborough. The owner, Mamdouh, is a great guy.

If you’d like to know what’s up in your community, and if you read blogs, you should check out Localfeeds.com, which works in cooperation with GeoURL. See that green GeoURL image at the bottom of the screen? If you click that, it will show you other blogs run by people within 25 km of my Toronto Apartment. Mapping virtual spaces to geographic locations is cool :) Anyway, Localfeeds is sort of a GeoURL aggregator or sorts. Speaking of aggregators, this site shows promise, but lacks implementation :(

Comments (3) Trackbacks (0)
  1. James Trott (UK)
    10:39 am on December 4th, 2003

    How do you do it derek, always switch me on to some cool new technology! How do you find this stuff? HOW!!! Ive got big big stuff coming… Gotta work on my damned site! Need to start spreading the word of cool, ive filled 2 whole notebooks with info to add to blogs and share!

  2. derek
    11:11 am on December 4th, 2003

    For you web-dev-heads out there, check this out! — It shows you what your page would look like to a search engine or spyder. Just follow that link and click “Diagnostic View” to see this page the way a search engine would. Then type in your own urls! Great tool.

  3. mom
    7:52 pm on December 4th, 2003

    how do you figure that $25 can is cheap when you can buy a dvd player for $79-$250 can depending on what you want/need?. I guess it depends on how many movies you watch, i never have 2 hours to sit and watch one. how are your eyes doing? hugs

Leave a comment

No trackbacks yet.

Comments (3) Trackbacks (0)
  1. James Trott (UK)
    10:39 am on December 4th, 2003

    How do you do it derek, always switch me on to some cool new technology! How do you find this stuff? HOW!!! Ive got big big stuff coming… Gotta work on my damned site! Need to start spreading the word of cool, ive filled 2 whole notebooks with info to add to blogs and share!

  2. derek
    11:11 am on December 4th, 2003

    For you web-dev-heads out there, check this out! — It shows you what your page would look like to a search engine or spyder. Just follow that link and click “Diagnostic View” to see this page the way a search engine would. Then type in your own urls! Great tool.

  3. mom
    7:52 pm on December 4th, 2003

    how do you figure that $25 can is cheap when you can buy a dvd player for $79-$250 can depending on what you want/need?. I guess it depends on how many movies you watch, i never have 2 hours to sit and watch one. how are your eyes doing? hugs

Leave a comment

No trackbacks yet.

Lernin’ Weekend

1
Dec/03
1

new site features: now i get email when you comment! so i can reply to you even if it was an old post i wouldn’t normally be re-visiting!

Also – steve sayer got spam yesterday, which said something about derekmartin.ca in the subject line and message. Apparently they’ve scraped email addresses off my blog. SOOO, now I’ve added a feature where you can only see other people’s email if you’re logged in to derekmartin.ca. You can still add comments without logging in though.

I wish I knew how many hits derekmartin.ca got in university and college, and my previous hosting provider before this one. As it stands, I only have the last 3 months of information. So, when I added the page hit counters (see bottom of screen), I had to start them at these lowly numbers, when I know my previous slashdotting (for my shift.com articles) would have added substantially. Oh well. Thanks to Frank for linking to those free hit counter places on his site.

cool: Bored of reading the internet, or posting text to your blog? well, now you can phone the internet! Audlink.com allows you to post mp3s to your blog by calling a phone number! Best part? It’s a free donation-based service! Worst part? I bet they don’t have Canadian or Croatian access numbers.

weekend: On Friday I had a long skype call with feti from mojavi.org. He’s based in Ohio, and it was clear as a bell with no lag! (except for some microphone-related static). Unlike past versions of skype, the latest (0.95) had no sound-drop outs and no dropped calls! So, get the latest version!

Last night I had my first skype with my Mom since I got here. We email a lot, but since I haven’t bothered to get a phone, we haven’t phoned. Happily, she now has broadband internet — so she downloaded Skype and we had a chat :) Initially it was static-y, but then she just held the mic closer to her mouth, and it was super-clear, and Freeeeeeeeee. Good stuff :) Apparently the christmas lights are up on our house and the holiday season is shaping up nicely :)

I also spent a huge portion of the weekend learning stuff, which I love. If you ever need to know how to do something in PHP, ask me, cause I probably have a tutorial on the topic .zipped and ready to go. I always save good tutorials from the web, so I don’t have to find them again.

mark & frank — this shit will blow your minds. Pilgrim 3.0? After you look at that link, check all the free related code here.

I learned how to create dynamic graphs & charts in PHP. So, here’s one for you, based on derekmartin.ca karma rating data. This is accurate since I started posting karma with my blog entries.

karma breakdown

If you can’t see the graph, let me know and I’ll change it back from .png to .jpg format. Graphing data is fun. I didn’t want to clutter the legend, so Great is Karma >5, Good is 0 to 5, Okay is -5 to 0, and Bad is less than -5. I have great plans. I wish I knew how to do this in 4th year university, instead of using Statistica.

I also looked back into the open-source effort to do Flash Remoting with PHP (it’s only designed to work with asp & coldfusion). Well, turns out AMFPHP hasn’t been updated since the last time I looked — in July. I thought “there’s got to be a Reason… What is the reason?” — another site had sprung up, with an easier way to make it all work. Easier if you know how to use object-oriented php, which I do. So, Ghostwire is my new friend. I’m looking into this because currently when you search my site, you get a bunch of incorrect search results, even though I’m using the Google API to do my searches. Why is this? Becasue of the random content in those boxes at the bottom of the screen. Google indexes that content, BUT since it’s random on every page, when you follow my search link, that content is no longer there — it’s some other random content. Hence, it appears that ‘my search’ is broken, when really it’s Google indexing my random content — and there’s no way to prevent that — EXCEPT to make the random content un-indexable. How? I’m either going to pull it into flash, or write it using javascript. It’s unclear to me yet whether or not Google will index document.write statements from javascript, so I’m leaning towards flash for the random content.

I also saw a *crappy* tutorial on making your own Custom 404 Page in php, over at phpfreaks.com. I already have a much better Custom 404 page working, and it even does intelligent re-routing. What does this mean? I’m going to write a tutorial for phpfreaks and send it in. Or maybe I’ll send it to DevShed, which is my favourite code-oriented site. Their tutorials all have a “print pdf” option, which means you don’t have to go through them page by page. You can right click the print pdf link and “Save As”. Great stuff!

I also wrote a single-file image gallery, which I’m going to release as open-source code for people who don’t know anything about php. You put the file in a directory, and you tell it what directory the images are in. It does the rest (creating thumbnails etc). You can edit the stylesheet if you want, and it’ll alter the appearance completely. I also re-wrote my file upload code, and it’s much cleaner now. I guess I’ll post that on here too, soon. I’ve been working on DerekMartin.ca Version 5 — which will look much like this site, but have better navigation and ‘mini sites’ with their own navigation. One of those mini-sites will be my free code, and links to great tutorials.

idea: if ORME (orbitally rearranged monoatomic elements) is a cheap superconductor to create, can I create ORME wire, to use in creating better/the best electromagnets? Might it help a lifter lift?

Finally, my friend Sacha is in Hong Kong, and it sounds/looks *crazy*. He sent me some .jpgs :) I think I’ll ask him to post some updates on his TIG Updates page ;)

Tagged as: ,
Comments (1) Trackbacks (0)
  1. derek
    8:55 am on December 1st, 2003

    Ben Skinner just sent me this crazy link. The guy makes a different t-shirt every day, and wears it only for that day, then archives it. He has been doing this since Nov. 2001 !!! How much does he spent on t-shirts! *ALL* of the shirts are online, so you can see/read them. He also has an email list, so you can be emailed what today’s shirt says.

Leave a comment

No trackbacks yet.

Comments (1) Trackbacks (0)
  1. derek
    8:55 am on December 1st, 2003

    Ben Skinner just sent me this crazy link. The guy makes a different t-shirt every day, and wears it only for that day, then archives it. He has been doing this since Nov. 2001 !!! How much does he spent on t-shirts! *ALL* of the shirts are online, so you can see/read them. He also has an email list, so you can be emailed what today’s shirt says.

Leave a comment

No trackbacks yet.

Get Adobe Flash playerPlugin by wpburn.com wordpress themes