I need to store several millions of string with a length from 20 up to 700 characters. The sources is UTF8, so I need to create an UTF8 column to store it.
I know MySQL has limitation in the number of bytes it can index and this is also affected by the character set, so I get errors when I try to create an index on this column.
As I'm almost sure I'm not the first trying to do this, I guess there might be some known tricks to workaround this. One I came up with is to use a hash function and use this for the key, the problem is that no hash function will guarantee a unique key.
Any suggestion is very welcome.