I have them both installed and running on my laptop running Win 2K Pro. No problems so far. Provided they are on different port numbers everything seems OK. Just don't try and use Sybase isql to connection MS SQL.
Creful if it is SQL Server 6.5, it used to be a ditto of Sybase ASE and all the binaries are named same as well.
If it SQL Server 7 and above ensure 4that you set the upper memory limit of SQL Srever, with dynamic memory management there won't be any memory left if you want to start ASE after starting SQL Server.
I installed both sql server and sybase .It's not problem. if you use command line on sybase .You can use isql for sybase and osql for SQL .
If you have problem about isql . you use under directory x:/sybase/bin
or move isql to system32