Trust issues between workers and their bosses and lack of supervision are among worries that managers expressed