I still need to be able to delete in both tables. Some times I delete records in table A (and needs corresponding data i table B to be deleted as well). And some times I delete records in table B (and needs corresponding data i table A to be deleted as well).

Maybe I don't understand your proposal correctly, but the foreign key with cascading deletes seems to me to be working the same ways as a delete-trigger - and therefore only one of the ways.