You can set the 'file size limit' to unlimited. If your /workspace is not having enough space to accomodate the backup file, then obviously you will get an error.
Still if you want to restrict each backup file to smaller sizes, you can use multiple directories, eg,'db2 backup db to /workspace/d1, /workspace/d2. /workspace/d3.....'
Regards
Prakash