This is part 1 of a 3 part series.

1. use Switch; - This has got to be the worst implementation of switch/case that I've ever seen. Everyone else uses automatic fallthrough until it hits a break;. Not Perl! It not only uses last; instead of break;, but it also prohibits fallthrough unless you specifically ask for it or implement it using next;. The reasoning behind this? Switch uses the regular expression parser for its arguments instead of having one case line per value like every other language that implements switch/case.
   As if the current implementation isn't bad enough, Perl 6 is set to use a different set of keywords: given and when.
2. Line noise - Perl has an overabundance of variables that use punctuation. For example, there's an operator <> that reads for the current filehandle (defaulting to STDIN). $_ is the "magic" default variable, $/ controls what <> thinks ends a line, setting $\ makes print act like Java's println, $! is the last error from the OS. The use English; module mitigates this problem somewhat, but it is still a pain.
3. for/foreach - These functions do not need to be synonyms of each other. I've actually seen people who claim that for my $object (@objects) is the proper way to write that. It isn't! foreach my $object (@objects) is! This code, while not directly translatable into other languages, is more intuitive for people reading the code.
4. Things that should be in the core aren't - use Cwd qw/getcwd/; is rather dumb. cwd/pwd is a common filesystem operation, yet it is not part of the perl core.
5. use constant; - use vars; has finally been changed to our. Why hasn't use constant also gone away? I think having const WHATEVER = 5 would be a great idea.
6. mod_perl - Great idea, poor implementation. PHP CGI and mod_php require no code changes to work correctly even on horribly bad code. Why does mod_perl?
7. Perl 6 - Copying Java. Switching to a VM? Changing the object delimiter to . ? Why not change the string concat operator to +? Oh, that's right... you can't do that in a weakly typed language. Instead, it's changing to... oh wait, that's right... it keeps changing. It was _ last time I checked, but I bet it changes again before Perl 6 is finalized. The sad part is... Duke Nukem Forever will probably beat Perl 6 to market.
8. Arguments to subs - Perl subroutines do not allow named arguments, instead placing all arguments in the magic array @_ or hash %_. Function prototyping only allows you to specify the types of arguments.
9. elsif - Why is this named elsif and not elseif or else if?

That's all I can think of for now. I'll probably add more later as I work on the others.

Edit 1: #2 has been updated, #8 is new

Edit 2: #1 has added commentary, #9 has been added