Hi, there! I'm not there myself yet but I've looked into some of this a little. First, I would guess that what you want is an AMP, Apache + MySQL + PHP or similar system. FileMaker kinda does all of this in one tight little package, but assuming it's an AMP you want, you neeed Apache to serve the web pages with the PHP module installed/activated. PHP code will alllow you to interact between web pages and the MySQL database. All of these are free, fortunately, and have lots of documentation.
If you use Mac OS X Server 10.2, a lot of this is already pre-installed and pre-configured... I think there are some comment removals you need to make in the Apache configuration file (etc/httpd/httpd.conf?) but make sure you have explicit instructions before tinkering with that.
If you have a flat, sequential db in FileMaker, you shouldn't have much trouble bringing your data into MySQL... forget scripts and stuff. If you have a relational db, each FMP db would be a MySQL table within a single db.
If you want to continue using FMP as your db, you can still integrate it with Apache. I think there's a plug-in you can get, too, for php with FileMaker. If you want to stick with FMP but want more web functionality, look at Lasso. It will let you keep your existig db and web pages but add more (they made CDML).