In my previous blog entry, "Advanced Benchmark Analysis I: Yet more white-space trimming", I left you with the thought that our benchmarks changed with changing input.
This article shows you how to analyze those changes and how to draw conclusions from them.
Read this from beginning to end, and you then tell me with a straight face that it hasn't completely changed your life.
I believe it is high time that we stop dilly-dallying around, that we face the future for what it is, for what is was meant to be, and take the challenge.
for 0..3 -> $even, $odd {
say "Even: $even \t Odd: $odd";
}
An appropriate theme song could be "Start of Something Beautiful" by Procupine Tree (which just happened to be playing on RadioParadise twice while reading and then re-reading the link above on alternating even and odd days).
It turns out I’ve done to my blog what I swore not to: Stop updating it. However, I’ve also sworn that if I did I would come back to it and not give up.
So what happened?
Well, it’s been quiet here because the heat turned up a few notches in my day job, and the opportunities to actually apply psychological methods turned plentiful. I’ve been involved heavily in recruitment in a (the) major Perl employer these days, and while I’ve learnt plenty about the minds of computer programmers, I also find myself in the situation where there’s correspondingly little I can write about it. On one side because there’s limits to how much detail I can write about before giving out information best kept confidental, and on the other side because some parts of a recruitment process needs to be kept inside the company to not give candidates unfair advantages (or disadvantages).
Now in a related turn of events, I seem to be heading to the Nordic Perl Workshop 2010 and I’m thinking about putting together a talk introducing the idea of using methods from Psychology to Perl programming. Alternatively just a general light-weight something about some subject from the world of Psychology of Programming. Which leads me to, if anyone who’s been reading the blog still follows it, what was your favourite post? Or what post would you like to seen elaborated on? Or what would just make a good talk?
Or to put it like the quintessential computer/psychology crossover, ELIZA, would: Come, come, elucidate your thoughts!
En lisant le dernier « PragPub: The First Iteration » (le n°9, disponible sur le site de PragProg), dans l'article « JavaScript: It's Not Just for Browsers Any More » de Jason Huggins, je suis tombé sur une réflexion intéressante :
When we choose a technology to write an application, we don't just choose the language, we also choose the list of available libraries. If a language has many useful libraries with a vibrant community around them, it's going to be easier to write your application in less time.
Bon, dans mon cas, cela m'a fait pensé au Perl et au CPAN, mais je suis sûr que d'autres liront autres choses :)
So, I got a Macbook Pro from $new_company (about which more later) and, as I believe is traditional, here are some notes about the experience.
Very nice: no rebooting, things just work from more or less the moment you take it out of the box. (Compare with recent Samsung N140 netbook, whose Windows 7 install wanted several reboots to install all the additional antivirus and other crap it comes bundled with.)
My parents, brother, gf, and her mother all have macbooks. And you have to admit that out of the box they come with a rather well thought out set of applications for real people. But of course everyone has different needs, and I immediately installed the following:
(Some are payware, eek! I'm trying them out for 30-day trials, and may get some through work, others I might be persuaded to pay myself, but lets see)
Then some file storage apps:
And some toys for the menu bar:
And miscellaneous bits and bobs:
I've not had time to play with all of these, but the overall quality is pretty good. Installation isn't too bad, though I don't think it's intuitive to "just drag the icon to the applications folder". You can tell it's not intuitive by the fact that every .dmg has a different background image with a large arrow graphic trying to point it. Or by the fact that my parents launch the one or two applications they've downloaded by opening them from a .dmg that they have permanently mounted... sigh...
Ah, there's even an article about this.
But OK, once you know how to do it, it's easy enough. Though I'd like apt, and should look at MacPorts. I did install brew which is nice for command-line apps.
Hardware is nice. Mostly. It doesn't run too hot. I like the lighting on the keyboard. Actually the keyboard is much nicer than I'd expected (One of the few really excellent things about the thinkpad was its keyboard, so I wasn't really expecting all that much from these odd scrabble-keys, but I'll talk about that in more depth shortly).
The new Mac trackpad is OK. I think I'd rather have a button - it's not as if you can click anywhere on the trackpad to click, it has to be near the bottom, so why not just have a button? On the other hand, tap-to-click works fine. Not having physical buttons means that there isn't a middle-mouse button. Which is inconvenient in an ubuntu VM, as it means there's no obvious way to do copy-paste in an xterm... I worried about this for a while, then came up with the obvious workaround instead -- use gnome-terminal... (who needs unicode, colours, or stability anyway?)
The trackpad seems insensitive too for "click; move; click" sequences. OK, this is mainly a problem for FreeCell and DiceWars, but annoying in any case.
The screen is nice too. Oh, the screen: this was largely why I had a sudden wild shift from a 12" supposedly "ultraportable" thinkpad to a 17" MBP... I decided that I couldn't see enough on a small screen. OK, so the 17" is massive... embarrassingly so when you take it out on a train for example, but it does make for a great machine to work on.
There are a couple of things I'm not so pleased with though: though the machine doesn't run hot (like my thinkpad) it does tingle in a semi-electric-shock way when the power adapter is plugged in (a quick google suggests this is a fairly common problem). Also the headphones, while loud and crisp, seem quite noisy. When you have the phones plugged in, every time a noise occurs, the headphone socket then hisses for half a minute, then switches itself off again.
The most annoying thing about the keyboard is the odd positioning of the keys. The # key is hidden away in Alt-3 (but without anything written on the keyboard to hint that). Oddly under my ubuntu VM (with keyboard set to Apple MBP) it insists on RightAlt-3. PgUp/PgDn are Fn-Up/Fn-Dn, which is reasonable but unexpected. Also, there's only one delete key, which behaves badly in Terminal.app until you set the appropriate options correctly (why isn't that the default?). Yet, there are some oddities taking up precious keyboard real estate:
Also, instead of Ctrl being at the very bottom-left (a lovely position, which means you can press Ctrl without using a finger -- I use the pad of my left hand) the "Fn" key is there instead. The Thinkpad shares this idiocy. I've recently been using a 5-year-old HP laptop as a backup, and despite it being underpowered and falling apart, the 2 things I really liked, after using a thinkpad, were a) the Ctrl key being in the right place, and b) that it had a trackpad. (The Thinkpad just had a clitmouse. After 2 years I learnt how to use it, but still found it hateful. Never again. But I digress.)
Another oddity is the media keys. (Where by "oddity", I'm struggling for a politer word than "batshit crazy clusterfuck").
I never really got media keys working satisfactorily under Linux, where they mainly just spoke to the foreground application. So I was hoping that a bit of Apple shiny would go a long way. Now I'm not a usability expert, but I'm hoping that my expectation for what a Play/Pause button should do might not be entirely insane or impractical:
What actually happens is this:
So, if you have VLC open playing a video, and somebody calls you, you press the Pause button. Then:
But that's OK. You can press the Pause button to stop the chaos right?
Apparently this is breakage from 10.6 Snow Leopard, having been sane till 10.5. There is speculation that Apple are trying to make 3rd party apps look bad by not responding "properly" to the media buttons, while Apple's own apps do. If that's true, then it's not working. I don't blame VLC. I don't blame Miro. Dear Apple, you fucked up, please sort it out.
(This isn't just me: various threads including http://discussions.apple.com/thread.jspa?threadID=2122639 are whining about this idiocy. The thread mentions a cunning back: if you have QuickTime open, it acts as a prophylactic against iTunes opening. This is apparently because the 'loginwindow' process has an exception hardcoded into it not to launch iTunes if it detects QT running. So not in the slightest bit insane then.)
I had the same issue with the remote. Yet it's shiny, but if it only wants to play with iTunes and the useless Front Row (where by useless, I mean doesn't play my videos, which is what I expected it to) then it's basically a shiny paperweight. Thanks Apple. I've installed Candelair, which may possibly help improve this but I've been disinclined to test properly.
When the machine resumes from sleep, the external monitor shows static "snow" for half a minute or so. That's not particularly impressive. A quick google suggests that older models had similar problems, perhaps this resurfaces every now and then? I should probably check to see if it's going to be a deteriorating hardware issue.
On the other hand, it actually resumes. From sleep-to-ram. And hibernate-to-disk. Every time. Yes, every time!
(Can you tell I've been using Linux-on-laptop for the last 5 years?)
And though I don't think I'm getting anywhere near the 8 hours advertised, I am to be fair running with a bright screen, a couple of VMs and so on.
This is a mixed bag. Mainly, the windows are less useful/pretty than Gnome: you can't set windows to Keep-on-top. You can only resize using the bottom-right control, no Alt-to-drag option. No ability to maximize a window (ok, so the need for this doesn't come up as often as I'd expected, but when it does, it's annoying), and new windows don't get tiled in pretty places as with gnome. Also, more often than I remember happening under Linux, popup windows get placed underneath the windows that created them. Meh.
I'm not sure whether I like Finder better than Nautilus. It seems to be possible to use drag-to-copy into a folder of folders (rather than into one of the subfolders). On the other hand "type-to-jump-to-filename" seems to not work consistently (I haven't quite figured which state is inhibiting this).
When you're using 2 monitors, Mac's single menu bar (on the top of the screen instead of top of window) becomes a royal pain, as you have to move the pointer from the external screen back to the laptop's one to use it. I guess I should learn the command keys to do this without mouse. Also, if you put the second screen immediately above the first, you suddenly change the menu bar from being "a mile high" to being a thin strip of stuff that you have to target. So don't do that (or learn those key combinations).
Spaces... wasn't sure how to move apps between spaces. But the F8 view is quite cute when you work out what it is (I was confused by it showing me what looked like a 4x2 view instead of a 2x2 one... but of course I have external monitor plugged in... d'oh.) Very cute is that Expose works while you're in the thumbnail view.
Date/Time applet in menubar is underpowered. Compare to the Gnome one that has a month calendar view, world date-time and weather. This one can't even be configured to show the date in the menu bar. It's a prime candidate for replacement with something actually useful.
The only thing I can think of that might explain the weak Date/Time applet is that they are trying to sell you on the Dashboard one? If so, it doesn't work. The Dashboard is a wonderful combination of ugly, confusing, and useless. Every time I opened it (by accident, mostly) I would struggle to either get it to do anything useful or, even better, close again. Luckily there is a 'defaults' setting that prevents it from opening again. Phew. Bizarre.
For dev work, I was planning to use VirtualBox, which has the nice feature of being free. It has a few problems though:
VMWare solves these, though, oddly, it actually seemed slightly harder to setup. Installing the "guest additions" had to download a large package, and instead of running it itself expected me to read the system documentation to know what to do with it (I just ran the perl script from terminal, which works fine... Not sure why the README file couldn't have just said that). Still, minimizing friction between guest and host working is probably worth paying for.
There are some niggles, and some outright baffling brokenness, but actually I'm quite enjoying working on a Mac.
I am happy to announce that the Haifa Perl mongers are going to have a meeting on the 15th March in the offices of Qualcomm in Matam, Haifa organized by Shmuel Fomberg. On the agenda is Erez Schatz How to Talk to Newbies and Yaron Meiry (aka Sawyer) Moose - A postmodern metaclass-based object system for Perl 5 The meeting will start at 18:30.
For more details please see the announcement and/or contact Shmuel.
The regular Rehovot Perl Monger meeting is going to take place on 16th March in the Weizmann Institute. Yaron Meiry (aka Sawyer) is going to give a talk about Moose - A postmodern metaclass-based object system for Perl 5
For more details please see the web site of the Rehovot Perl Mongers.
It’s never great to post an “end-February” report a week into March, but that’s how things are going lately. I’ve been busy with family and work obligations that have meant less CT2.0 hacking. I’m sorry this is coming late, but I hope I will give anyone interested a sense of where things stand.
I should note that the original deadline for finishing CT2.0 was March 1 and clearly we’re not there yet. I’ve discussed the situation with Robert and Ask at the Perl NOC, and they’ve been willing to extend the deadline for cutting off CT1.0 for a while longer. Thank you, Robert and Ask, for your understanding!
Progress in the last couple weeks:
In the last several weeks, members the #catalyst and #moose IRC channels were very, very helpful and patiently answered my many stupid questions. Thank you in particular to confound, perigrin, rafl, rjbs, stevan and t0m.
Coming up:
It’s at the point now where I suspect the “hard thinking” part is pretty much done and it’s a lot of grotty but straightforward tasks to go. Hopefully, beta testing won’t reveal any major issues and the end of March update will be focused on planning on orderly transition from CT1.0 to CT2.0.
Latest Web::Simple master (git://git.shadowcat.co.uk/catagits/Web-Simple.git) is now using Plack, the perl web server directly for CGI and FastCGI and I hope do do another release shortly with notes on deploying as both on shared hosts (more specifically Dreamhost but with an invitation to bitch if they don’t work on other budget hosts).
Every time someone mentions a cool new web framework like Web::Simple, Dancer or Tatsumaki, my immediate reaction is to start thinking up cool little niche web apps I could build with it. You know, the kind you imagine yourself whipping up during your lunch break, that will probably never get more than 10 curious users but just might turn into the next big thing if only you got it up and running on a public server. And that’s when the second thought immediately arrives: where am I going to run this thing? Do I really think the idea has enough legs to justify paying for a virtual server plan? Can I be bothered going through the pain of figuring out how to deploy it on a shared hosting provider? And normally that’s the point where I sigh wistfully and go back to reading my RSS feeds.
The thing is, with the advent of things like local::lib it’s getting a lot easier to deploy Perl web apps on shared hosting. And with most web frameworks adopting Plack/PSGI, the work required to deploy Perl web apps on budget hosts is converging into a similar sequence of steps.
So this time during my lunch break I started envisaging a centralised, SEO-friendly information source (hello perlsharedhosting.com) that cobbles together all of the currently available information into an easy to digest form to make it ridiculously easy to choose a shared hosting provider, deploy your web app and troubleshoot common problems.
A site that works like this:
I haven’t decided yet what engine would fit best.. MojoMojo is a front-runner, and in fact the Catalyst Friendly Hosting page runs on MojoMojo does a large chunk of what I’ve described above.
The site would be very “cookbook” oriented, since we’re specifically targeting people who can’t be bothered learning the ins and outs of 10 different technologies (not to mention figuring out how to get them to play nicely together) for the sake of deploying toy web applications on cheap shared hosting. Just the essentials: this is what you need, this is how you achieve it. And if you want to learn more, go here.
With that in place, we’d end up with a single, visible, place to research and document the complications of running Perl web apps on shared hosting. And if the site became popular, hosting providers might even take notice and start offering more Perl-friendly shared environments. We could have live demo pages running on different hosts, possibly even donated by hosting providers if they see it as a way of showcasing their Perl-friendliness (mojomojo.dreamhost.perlsharedhosting.com, dancer.resellerzoom.perlsharedhosting.com, etc..).
I got as far as registering the domain name; I was planning on getting a simple MojoMojo prototype up and running, but I got side-tracked researching how to deploy it on my shared hosting account…
A new version of Dancer has been published to CPAN a few minutes ago, it’s also available from the official website, in the download section.
As you can see, the project has now its own domain name: http://perldancer.org, it’s clearly more shiny than the previous subdomain I used ;) So feel free to update your bookmarks (even though the old domain name now redirects to the new one).
Oh, and there is also a mailing list now, for people who like to share experiences about Dancer; feel free to join.
The change set for 1.160 is impressive, thanks a lot to everybody who got involved!
Happy dancing.
We were talking to many PHP guys on CeBit 2010. I tried showing the huge power of Perl but – as far as I remember – I never said anybody “forget PHP”. Here are the most important items (odered by usual order of appereance):
Neither PHP not Perl people like that fact, but PHP was originally based on Perl. There are still many similarities between both considering keywords and syntax. But most PHP people agree if you say: “If you know PHP, you basically know a good amount of Perl.”
There are about 20.000 modules for all kinds of things on CPAN, like database access, file parsing, templating and math and all of them are free. All CPAN modules could be installed by a CPAN client which usually comes with your Perl.
Please tell me if you know any other language which has something comparable.
Most of the PHP users also use MySQL but for the full time of CeBit 2010, I met only one person (who actually was a database expert with some PHP knowledge) who answered the question “Do you like to write SQL statements, expecially SELECT JOINs over some tables?” with yes, all others answered between “no”, “not really” and “less than writing PHP”.
Basically DBIx::Class abstracts the SQL statement layer: Your source dosn’t need to contain one SQL statement any longer. The results are:
Nothing written above has anything to do with comparing PHP and Perl. It’s just comparing DBIx::Class to $dbh->select… (or other ways sending SQL statements in other languages). There is something on PHP which might be compareable to DBIx::Class even (up to me) it isn’t nearly as powerful as DBIx::Class, but most PHP – people I met on CeBit 2010 were still writing plain SQL statements – as I did about eight month ago.
Also remember that using DBIx::Class will introduce a slight part of OOP even to people who don’t actually use it. Up to me this would be the preferred way of starting OOP development because it’s a slow start – you could add more and more OOP as time goes by and you feel ready to look into another world.
There are technologies for PHP which are really better in many situations than these two. Catalyst is a complete Web framework which reduces the typical programming amount for weblications and Template::Toolkit (TT) provides a very powerful template engine, but I was working together with some webdesigners in the past which can’t handle it because it was too programmish.
If you pass a DBIx::Class object to TT, the one writing the HTML code could easily access all data of the table row including any child objects and any future changes to the table structure without any additional application source.
I wrote about three examples of the more than 20.000 CPAN modules, but if you ask people what else do they want to do and live-search for it on CPAN, it usually gives you some results. Please recheck this if/when the CPAN search engine (which shows modules even on very slow relations to the search) is being fixed.
Good samples to show are
Padre, the free Perl IDE comes with some few lessons within the examples, not enough for really learning Perl but good for looking into the syntax.
The next step would be http://learn.perl.org for many more resources how to learn Perl.
If you want to meet the community, look at the PerlMongers groups who meet maybe also in your area.
…you should at least try Padre which also has a PHP plugin for working on PHP files. You’re welcome to improve it by testing or changing the source.
Last notice: Writing Padre Plugins in PHP should theoretically be possible using the Parrot engine, if you want to try it, please add your experiences to the Padre Trac wiki.
I recently held a introduction to the Mojolicious web framework for Oslo.pm. Here are the slides:
CeBIT was way more successful than I expected. Thanks to Renee Baecker, the main organizer and the other people who were at the stand during the week we made a lot of contacts with individual developers and companies using Perl. We also talked to quite a number of people who have heard about Perl but never tried it and some people who never heard about it.
It was both very hard to be there standing 8-9 hours and almost constantly talking to people every day but it was also a lot of fun.
Originally we were requested to always have two people at the stand but it turned out that there were times when even 7 of the Perl::Staff members could not handle the number of visitors at the same time. We were lucky as our stand was part of the Open Source Lounge of 15 projects slightly out of the main stream of people. On one hand that meant many people walking along the main alleys missed it but on the other hand those who came by had more space and time to talk to us.
It was very heart warming to see people coming to the booth with no or even negative view on Perl and leaving with a feeling that they saw some really cool things they want to try now.
To people with now real Perl knowledge or people who used Perl 10 years ago we showed some modern Perl code such as Moose, MooseX::Declare and some related stuff. We showed them Padre so they can see you can write large desktop applications in Perl. We were also showing Catalyst, especially to PHP programmers but also to others who were web developers.
On one of the days there were two Foswiki developers and they showed their wiki to the people interested in that. I think OTRS got a bit less attention on our stand but they had their own booth very they made a lot of business contacts.
The strategy was simple. Some of us were standing in the alley and as people walked by we gave them a flyer and then started to talk to them. Obviously there were many who were not interested, we just let them go. I was the only presenter who did not speak German my strategy was to first ask them if they speak English. With many people it was clear that they don't know English well enough for a conversation or just prefer German. That gave me a chance to direct them closer to the booth to talk to "one of our other representative, who speaks German". So in fact I was acting a lot as the "catch man". I think it worked out quite well. When they were OK speaking English or even preferred that language then I kept the conversation going on. I asked what are they using Perl for, or if they did not know or have not used Perl I asked what other languages are they using and what kind of things are they using. That usually gave me an opportunity to ask if they would like to see some modern technologies in Perl. Most of them were interested so we stepped to one of the computers we set up (we had 2 sometimes 3 computers on our booth, which was high enough so we did not have to bend over to view the screens or type). There I showed them Padre, some code in Padre. Sometimes I showed a few pages from the Moose documentation and some small examples in Catalyst.
There were several people who were interested in Perl 6. For them I opened the web-page of my Perl 6 training slides and showed a few pages just to get them impressed by some of the nice features of the language.
Others, as I saw started by asking if the visitor "Knew Perl?" and from that point some kind of a conversation evolved in German that I could not follow anyway but that will be described by the other presenters. (Renee and Sewi have already written. All the reflections will be linked from the TPF wiki page related to Perl on CeBIT)
There were a lot of things we could improve in our presentation. First of all none of us knew all the things we wanted to show and we did not have ready made pages to show them. So I'd like to have several slide-shows that we can load on all the computers we are using for presentation and that each one of us can use to present various technologies. After all I don't need to be an expert in Catalyst in order to give a short introduction and show a few examples, assuming they were already prepared and I went over the slides with someone who is an expert.
Same with the Perl 6 examples. Others kept referring people to me to show Perl 6 examples but we could have put together a few slides with interesting examples that we can flip though pointing out the nice features. Within the slides we could even have instructions - show the Rakudo web site now, or in other cases to show the web page of the CPAN Testers.
This of course needs more preparation and even some time before the event starts to show the slides to each other but it pays off as it will make the presentation smoother.
Even that does not mean there won't be questions that need some more thinking on how to answer. There can also be cases when the knowledge of the presenter ends and there are still questions but then we can find the other presenter who has deeper knowledge in that subject or refer the person to the community channels where can get more information. After all, a large part of what we would like to achieve is that our visitors will start learning more about these subjects and start to participate in our communities.
Because I recently had the opportunity to do a fresh re-install of my world1, I've spent the last few days playing with App::perlbrew. App::perlbrew is the invention of Kang-min Liu aka gugod, and the basic idea is that it's a perl manager. It will install and track several different installations of Perl for you, allowing you to switch between them at will.
$perlbrew installed
perl-5.10.1
perl-5.8.9
perl-5.6.2
$perlbrew switch perl-5.10.1
$perl -v
This is perl, v5.10.1 (*) built for darwin-2level
I'd like to state now that what gugod had was very nice. It was very simple, very straight forward and did exactly what it said on the package. I have not spoke with him about this application, and anything I say here is my opinion and has no reflection upon him or anybody else that might have been involved.
Having seen what power some simple scripting can do in the form of Miyagawa's App::cpanminus, I started tinkering with perlbrew. It started with perl 5.10.1 not installing properly on Snow Leopard2, so the first thing I added was a way to force install.
perlbrew install -f perl-5.10.1
Then I decided it and local::lib both sharing $HOME/perl5 wasn't going to be pretty. So I taught it to use an environment variable PERLBREW_ROOT to relocate the default install. In my setup I have it set to $HOME/.perlbrew.
$export PERLBREW_ROOT=$HOME/.perlbrew
$perlbrew init
Attempting to create directory /Users/perigrin/.perlbrew/perls/current
Perlbrew environmet Initiated.
Required directories are created under /Users/perigrin/.perlbrew.
Please add this to the end of your ~/.bashrc:
source /Users/perigrin/.perlbrew/etc/bashrc
Then I got annoyed with the pages of verbose output, so I stole a page from cpanminus and implemented a quiet switch that is enabled by default. Output now goes to $PERLBREW_ROOT/build.log.
Today I integrated it with local::lib so that if you have local::lib installed it will drop the proper configuration in the perlbrew configuration scripts so that when you install new modules they get installed into your perlbrew managed directories3.
Finally just now I finished getting --as= working. This means that if you build a custom perl you can have it installed under a special name and then easily switch to and from it. I'm planning on using this to build a --as-workperl that contains all the modules I need for work.
$perlbrew install perl-5.10.1 --as=debugging-perl -D=debugging
I would like to integrate a plugin system4, but all in all I'm very happy with it. Much thanks to gugod for making it to start with, and to Miyagawa for getting me itching for small lightweight tools in my toolchain. If you're interested in my changes, you can check them out on my github
By: Chris Prather on 2010-03-07T00:00:00
Tags:
Last week I upgraded my trusty Macbook Pro from OSX 10.5 (Leopard) to OSX 10.6 (Snow Leopard), using a "complete wipe". The old system had been around for ~3 years and was showing the cruft so it was time. ↩
Apparently there is an issue with Snow Leopard's locales, and a single test fails. ↩
Full disclosure, I got this to the proof of concept stage but not really much further. It currently expects local::lib to be installed to $HOME/perl5 (which is the default). If you have it installed somewhere else, you'll need to manually set up the environment. ↩
Having recently played with plugins in cpanminus I have to say they're incredibly nifty. The github plugin is especially nice. I especially would like a plugin to allow swapping Git in for several parts of the system. ↩
So mothers keep your hackers at home
Don't let them journey all alone
Tell them this world is full of danger
And to shun the repositories of strangers
- The Tag Set of Strangers,
(with apologies to) Nick Cave and the Bad Seeds
One of the things I love about Git is how I can add branches from remote repositories in mine at will without fearing to mess up anything. The remote branches will not clash with my own, even if they share the same names, because they are referenced as repository/branch. However, as for anything else, you can still poke yourself in the eye if you try hard enough:
$ git remote
bob
$ git checkout -b bob/baroque
Switched to a new branch 'bob/baroque'
$ git fetch bob
From ../bob
* [new branch] baroque -> bob/baroque
$ git checkout bob/baroque
warning: refname 'bob/baroque' is ambiguous.
Already on 'bob/baroque'
Here, I created a local branch called bob/baroque, which will end up having the same name as the branch baroque imported from Bob's repository. Confusing, but not the end of the world. I can still see the difference branches with gitk and access the local and remote branches via git checkout remotes/bob/baroque and git checkout remotes/bob/baroque.[1] The lesson to take out from this, of course, is simply not to use slashes in branch names and sidestep the whole issue.
[1] The fiendish-minded reader probably wonder at this point what would happen if I was to create another local branch called remotes/bob/baroque. I would deserve to be shot, that's what would happen.
With tags, surprisingly, the matter is much more prickly. Not only tags are not kept to the ''namespace'' of their repository of origin, but git fetch has a very dangerous default behavior:
$ git log -n 1 somework
commit 483d008c6207554236232fef4e8cd22cfb4b9bb8
Author: Yanick Champoux <yanick@babyl.dyndns.org>
Date: Wed Mar 3 21:14:43 2010 -0500
some work on my repo
$ git fetch --tags bob
From ../b
- [tag update] somework -> somework
$ git log -n 1 somework
commit 5f7f8eddd2d44e359fe8bc0d1a2f1642d073cad9
Author: Yanick Champoux <yanick@babyl.dyndns.org>
Date: Wed Mar 3 21:15:25 2010 -0500
some work from Bob
Yes, if there is a conflict, fetch --tags will silently clobber the local tags with their remote counterparts. Hope you remember all the commits you painstakingly tagged in the last six months... This behavior is so mind-bogglingly dangerous that, to this day, I wonder if I'm not missing something obvious.
Now, granted, it's fairly rare to import tags from remote repositories. But there are instances, like when adding the gitpan history of your module to your repository, where it's relevant. In those cases, for your own sake, make sure that the remote tags won't clash with yours. Even better, don't use fetch --tags at all. Instead, do something akin to:
$ git ls-remote --tags bob | perl -nae'$F[1]=~s#refs/tags#bob#; `git tag $F[1] $F[0]`'
This will name remote tags using the repository/tag convention. Even better, if the tag already exist, Git will complain and it won't be clobbered by the new version.
A while ago I joined the Perl Iron Man Challenge to motivate me to write more regularly about Perl. I’m not sure how successful that has been. I was reading Leonard’s post about how he finds it hard to come up with a topic every week, but he has managed to do this for eight months. I think the longest consecutive run for me was eight weeks.
I should have things to write about as I am working on a variety of community projects. But lots of what I do isn’t overly exciting and I find it hard to be inspired to write about it. Even when I am really pleased about something, like Dave Mitchell’s grant being accepted, I can’t think of anything much to say. Maybe someday I’ll get better at this.
There is more than one way to handle forms in your Catalyst/DBIC application. This paper speaks to the HTML::FormHandler way. Let's start with a simple example of building a registration form to allow new users to register with your website application1. We will look at how to build a registration form object and use it in a Catalyst controller. Furthermore, the form will integrate with a DBIC connected database to persist the registration information submitted via the form.
We build a registration form class that uses the DBIC model like so:
package Bracket::Form::Register;
use HTML::FormHandler::Moose;
extends 'HTML::FormHandler::Model::DBIC';
with 'HTML::FormHandler::Render::Table';
has '+item_class' => ( default => 'Player' );
has_field 'first_name' => ( type => 'Text' );
has_field 'last_name' => ( type => 'Text' );
has_field 'email' => (
type => 'Email',
required => 1,
unique => 1,
);
has_field 'password' => ( type => 'Password' );
has_field 'password_confirm' => ( type => 'PasswordConf' );
has_field 'submit' => ( type => 'Submit', value => 'Register' );
has '+unique_messages' =>
( default => sub { { player_email =>
'Email already registered' } } );
no HTML::FormHandler::Moose;
__PACKAGE__->meta->make_immutable;
1
Notice that we're extending the DBIC model so we can easily create records in our players (think users) table. The '+item_class' attribute is where we specifiy the DBIC Result class that we bind the form to. In our case this is the Player class that corresponds to the player table in our database. Player is synonymous with the more commonly found user table where one has a username and password column.
FormHandler has a Moose-like attribute declarer has_field. This allows one to declare fields where the type is a HTML::FormHandler::Field such as 'Text', 'Email', 'Password' etc. These type declarations enable cheap and effective validation. For example, a field with type Email will be checked using Email::Valid. In addition we set the email field as required and unique.
I've asked for the rendering to be in table form with the 'HTML::FormHandler::Render::Table role.
In order to have an instance of the registration form object available in a Catalyst controller I do:
use Bracket::Form::Register;
has 'register_form' => (
isa => 'Bracket::Form::Register',
is => 'rw',
lazy => 1,
default => sub { Bracket::Form::Register->new },
);
Later in a controller action I will be able to access the form via $self->register_form.
sub register : Global {
my ( $self, $c ) = @_;
# Stash the form and the template to render it
$c->stash(
template => 'form/auth/register.tt',
form => $self->register_form,
);
# Create an empty row object for the desired table
my $new_player = $c->model('DBIC::Player')->new_result( {} );
# Process the for with the parameters, a schema and a row object
$self->register_form->process(
item => $new_player,
params => $c->request->parameters,
);
# This returns on GET (new form) and a POSTed form that's invalid.
return if !$self->register_form->is_valid;
# At this stage the form has validated
$c->flash->{status_msg} = 'Registration initiated';
$c->response->redirect( $c->uri_for('/login') );
}
Here we have one action that handles both the GET and POST request methods of the form. On GET, HTML::FormHandler will see that there are no parameters to process and just return the form. On POST, it will check that the parameters are valid and then process them accordingly. In our case this means, create a new user record from the form fields where each form field that corresponds to a column in our player table will be used to populate the column.
We start the action body off by stashing the form and template that will be used to render it. All we need in the template2 is:
[% form.render %]
Since we want to create a new record in our Player class, we create a new row object of that nature with DBIC's new_result() method3.
Next we process the form with the following key/value combinations:
Note that since we're passing in a DBIC row object as our item the schema is inferred. If one is passing in a primary key this can be done by passing in the primary key as 'item_id' and additionally pass the schema. For example:
$self->form->process(
item_id => 1,
params => $c->request->parameters,
schema => $c->model('DBIC')->schema,
)
The form is then validated and returned in its orginal form if it's not valid. Otherwise we have a successful registration and proceed to login5.
FormHandler enables one to efficiently define forms and bind them to a DBIC model. In addition, it allows types to be declared on fields which are then used in validation. It also supports other form field properties such as: unique and required. In short, it does all that is needed for basic form handling without having to write a single line of HTML.
1 Instead of the proverbial 'MyApp', I'm using the application name 'Bracket'. This is for a NCAA tournament bracket app. I'm cooking up. Ping me if your interested in it.
2 We're using a Template Toolkit view.
3 The row is not actually inserted yet, just prepared to be.
4 When we pass a row object item to process() we don't need to specify the schema.
5 An upcoming article will discuss how to build a login form backed with authentication in Catalyst using FormHandler and DBIC.
Well after what seems like several decades I am finally going to Review 'Intermediate Perl by Randal L. Schwartz, brian d foy and Tom Phoenix. The book is on the whole very well written and has an amusing style which belies its importance. Initially I read Randal Schwartz and Tom Phoenix and brian d foy which gave me the basics, enough in fact to use some cgi scripts that, although now need a re-write to due my added knowledge, are still in use today.
I then went onto Programming Perl by Larry Wall, Tom Christiansen, Jon Orwant which I read as a sort of novel, when I told this to Larry Wall at a YAPC he said that it was mostly fiction anyway. It introduced me to alot of the advanced features in Perl but was a bit heavy going. After I had read it I wondered if there was something inbetween this and Learning Perl, something intermediate.
I know it seems obvious not but I did not look for a book called Intermediate Perl. I found this sort of pattern happening a lot in my jourmey through Perl, the obvious is so obvious I miss it. I have tried to start looking for the obvious before I hare off and look for the complicated and down right obscure which always seem more easy to see. As I had not cultivated the culture of 'The Bleeding Obvious' I bought a few others which, although good in their way were not really suitable at the time for where I was at. Eventually I got a copy of 'Intermediate Perl' and it all made more sense.
I read another review online which suggested a different order in which to read the chapters. The suggested thing was to read Ch 3 after reading Ch 10 and then read Ch 15 because it was thought that you need to know about modules after learning about building larger programs. This I did and it seemed to make more sense. Apart from my normal gripe about not explaining how to get help through Perldoc and a few examples which when typed in verbatum do not work (bottom of page 4 in particular its is not that it is wrong but to make it run you have to do a little more than is written) the book helped alot. This kind of thing is not a problem half way through the book, any book, a little assumed knowledge has been learned, but it is important to make things absolutey implicit in the first few examples. Implied knowledge is an easy way to dishearted anyone and could in extreme cases cause someone to give up. Reading it through at least twice meant that a few things clicked eventually and gave me a sense of achievement when I understood the concepts and put some of them to the test. The story of Gilligan, Skipper, The professor and crew meant a simple but powerful thread kept the example real. Although imagined, these examples explained in fairly simple terms some very complicated ideas that I found fairly easy to apply to real world applications. Having read 'Programming Perl' first meant that I had gone a long way to understanding many of the ideas but anyone going about it in the right order would have less of a virtical wall to look at when they got to the 'Camel'. I use Intermediate Perl as a reference at the moment and to re-read certain chapters when I have a quiet moment.
I would recommend this book to anyone who has just got through Learning Perl or who is coming to Perl from another language and wants to start in a comfortable place before scaling the heights.
Whilst reading the chapter on references I was struck by the one of the many lighthning strikes out of the snaffed and blurreddy that is 'The Bleeding Obvious'. It may be called a reference but I thought that that was just its name. Unlike a the fact that a variable is called a variable because its value can vary but not its declared name,(I know that that is not necessarily the case as it is its address in memory that does not change or perhaps something else that I don't know yet, but humour me) a reference is called a reference because it is a reference to some thing, usually a scalar, array, subroutine or hash or something. As a reference it refers to something. You cannot believe how I wondered how I had managed to get through at least 3 rather large books before it finally sank in. I am not sure if it was the fact I did not read the right books in the right order or the fact that I was not really reading what I was reading. Either way, I have had the same experience several times. I think that because it has always been pointed out that Larry Wall is a linguist and therefore has crafted Perl in a liguistic way that I assumed the opposite. I have the same experience with other languages and with other things too but that is part of the joys, or not, of not taking the obvious as obvious. Tell me one thing and I will often assume the opposite because that is what I thought you meant, and just when I think I have got it I will the think the opposite of the opposite which is of course the obvious but not everytime. Sounds dumb when you write it down but with everyone trying to be so damn clever I am not sure who is genuinely clever and who is trying to look clever by inference. In Larry's case I think he is just being genuine.
I will do a more indepth review of this book as it has helped me a great dealm but this will do to get started for now.
Seems my previous blog, "Some simple "white-space trim" benchmarks" caught people's attention, and I've received some interesting suggestions and observations worthy of a followup article, this also gives me the chance to delve into explaining more advanced benchmark analysis.
So, deep breath, here goes.
With Miyagawa's cpanminus (or as I like to think of it, cpantiny, but hey it's his module) now officially the New Shiny of the moment, the same old arguments are resurfacing about less is more, and worse is better.
And the great wheel of opinion circles again.
Rather than bother commenting on cpanminus itself (I suspect many could guess my opinions) I thought I would add some caveats to the praise, that hopefully can help you identify and engineer your own equivalent successes.
1. These "subset" modules are truly successful only when you make the accuracy trade offs worth it.
Thus, the installation must be effortless, the user interface must be super-simple, zero-conf is an absolute must, and the module must succeed in all those niches where the "real" application is too hard, too big, or too clumsy.
If your accuracy is going to suck, NOTHING else is allowed to suck.
2. As I hint about with "real" these subset modules work best when they are an alternative solution, not the only solution.
If you are inaccurate and the only solution, you are an annoying limitation.
If you are inaccurate and an alternate solution, you are handy because you create a kind of user-pays situation. Simple people with simple use cases get a simple solution. Hardcore people with hardcore needs get a hardcore solution.
So if you want to make a small solution, it has to be a subset of something larger that works better. It can't be the only solution.
3. 99% is a just a marketing number, but the real number does matter.
Why? Take a look at the Heavy 100 index (http://ali.as/top100/") and you'll notice that many major and popular CPAN modules (like, say, Catalyst) have more than 100 dependencies.
With a 99% success rate (using stupidly naive "statistics") every single module on that Top 100 list will fail to install. In a large system with lots of recursive dependencies, it doesn't take much for your install count to grow to 20 or 50 or 100 dependencies.
So you need to be sure about WHICH subset you want to support, so you can be sure what percentage you REALLY need.
For example, despite all the work to support it, there is only one single module currently using Bzip2. Would it be worth it to remove bzip2 support entirely from the CPAN and help that author to convert? Probably.
The biggest benefits of these alternate solutions are often not only to do what they do. It's to demonstrate what is possible, pushing the competitors to do it as well.
I proposed a grant for Dist::Zilla improvements to The Perl Foundation, and a notice of its approval was posted yesterday. I've gotten to work.
A number of the things I proposed to do have been on my mind for quite a while, but I haven't been able to commit to working on them. I had some of them already sketched out in branches that were unready to merge. Last night, I rebased and merged them, resolved a few conflicts, and tried to cut a development release. Unfortunately, I foolishly cut a production release.
Fortunately, nothing seems to have been horribly broken. A few missing prerequisites were exposed here and there, but those have been fixed. Some other, more serious problem were fixed tonight, and I cut an actual dev release. You can see most of my "to do" list in the git repository, if you're not content with the summary in the grant proposal.
So far, I've been checking out upstream test coverage, adding a proper logging facility, and changing the way prerequisites are registered. The logging changes are mostly to facilitate better testing so I can start writing tests for the entirety of Dist::Zilla as it stands now, before I get to any more refactoring or extension. The upstream coverage is mostly an issue because if we test lots of Dist::Zilla, but not the code written to make it go, that's a big coverage gap. I'd like to make sure that all the code (that I wrote) to make Dist::Zilla work is adequately tested. I want to be able to rely on this code, after all!
The prereq stuff has just been on my mind and in the margin notes of my recent
CPAN work, especially as it related to fixing some bugs in the excellent
AutoPrereq module. It may introduce a few bugs in the short term, but those
should clear up. It will also make it much easier to report prerequisites,
including build-time or configure-time prereqs. In another release or two, the
AutoPrereq plugin should be able to report libraries found in ./t as being
test requirements, separate from runtime requirements, for example.
When these tasks are complete, I'll be moving on to testing Dist::Zilla itself. That means writing some tools to make that easier, and probably a few changes to how Dist::Zilla works here and there, to improve diagnostics and configuration.
For now, I'm happy that I'm going to be able to spend some time on this. It's sorely needed improvement.
So you’ve graduated a fine university with a degree in math, economics, CS, or some other actually useful piece of paper showing you know (theoretically anyway) how to do something of use to society. Congratulations! At job fairs all over the country at the best schools, or perhaps the schools that company recruiters went to anyway, trading companies want to hand you the keys to a decent salary, possible ridiculous bonus, hot secretaries, and the best office digs money can buy. Oh, and a couple billion dollar trading limit if that’s what you’re into.
Should you take it?
Do you like money?
Do you like money more than anything else in the world?
That’s the question you have to ask yourself. You’ll make more than enough to live on. You’ll probably get a bonus ranging from a couple months salary to the type of money your parents wouldn’t be comfortable thinking about. The office will be wonderfully furnished, and the work will be interesting. They’ll only hire the hottest HR women, the prettiest secretaries. Free drinks, snacks, bar after hours, a car service, free insurance, paid breakfast lunch and dinner. It’s a crazy world.
It’s also all you’ll have.
Traders – you’re in it for money. That’s what you *do* after all. Play along for a couple of years, save your bonuses, and start a restaurant. Live a life of boring luxury if you want. IT folks, you’re my people. I don’t do it for the money – it’s sure as heck nice though. You probably do it because it’s coolest work you’ll ever find. You can use your beloved language (ahem, perl) because it’s what you’re the fastest and most productive in. The best, fastest, newest everything you can find. But you’re working for the traders. The traders make the company money, you are an expense. Sure, they give lip service saying that you’re all equal, but you’re not, are you? When they stay late so they can have more profits on their books it’s a measurable thing that shows up at the end of the year in their bonus. When you stay late, it’s so you don’t get yelled at. Uptime doesn’t measure to actual dollars.
- There’s no place in the world like the financial industry. The companies nearly literally do nothing but create money. Salaries are good, bonuses are better. I don’t know of anyplace in the world you can make as much. Period.
- Benefits match salaries. 100% paid health, vision, dental, AD&D, life insurance policies. Free food. Paid transportation – public, light rail, parking passes. Drinks and frequent outings and parties. Snacks and gym memberships.
- Figuring out low latency systems, writing programs that handle billions of dollars, tying systems made of dozens or hundreds of smaller programs together. New algorithms, different languages, the flexibility to change things. All make for time flying doing cool things.
- The smartest people money can buy! They’re not hidden in a think tank or back room either – they’re right next to you making your brilliant ideas sound silly and trivial in comparison. So many great people to learn from. Steal their book lists, read their favorite websites, pick their brains. Making you smarter makes the company money too, after all.
- Ever seen a trader’s station? The only limiting factor in the number of screens they get is their peripheral vision. Six 30″ lcd’s orientated vertically with 19″ monitors on top of that. Tech people get the hand-me-downs, of course, but you still get rather good hardware usually. Aeron chairs, anything ergonomic the office manager can find. Shiny toys bought by vendors for the company. Giant-screen TV’s everywhere. Kitchen to hang out in. Everything is beautiful and functional.
- Brand new routers, switches, servers. Fiber. Network cards with their own operating system. Fiber fabrics that use different frequencies of light to get more bandwidth. GPS time servers. At least gigabit everything. Terabyte storage arrays. Entire floors of a building designed around cooling. Datacenters. It’s really freaking cool.
- There’s no nicer way to put this one. You are saying that you value money over anything else. Health, morality, family. You’re only in it for the money, and the company you work for doesn’t do anything but make money.
- Ten hour days will get you fired for not showing up. The pace is frantic, at the end of the day your brain melts. Adrenaline from the awesome parts of the job doesn’t do it after a while. Sleep is for poor people.
- Yes, you can spend another year working to make your system trade faster. Just like last year. There isn’t even a target to shoot for. Just “faster”.
- The mighty Wolfram is famous for this. ”I’ll see you this weekend” “No, I’m watching the super bowl.” ”What’s a super bowl?” The people around you (and you, by extension) focus so much on the ways to make the company more money that nothing else exists. You have to be so good at what you do to stay ahead that that’s all you can focus on.
- See also You will burn out, above. They buy you for the hefty fees you are making. Wake up early to get in before market open, work, then study your job when you get home. We were actually planning to put convertible couches into the conference rooms for sleeping.
- Ever been to a casino? That’s a lot like the floor. You don’t even have the token courtesy of cubicle walls to stop the ringing, talking, yelling, sheep noises, etc. Add to that constant interruptions and a lot of work gets done. Badly.
- These companies are secretive, and for good reason. Everything from hardware, to network paths, to what programming language is fodder for the enemy. Trading is a zero-sum game, after all.
Hell yes. Or maybe not. Try it. If it doesn’t go against your ethics, you don’t mind taking a few years off from productive life, and can keep yourself broad enough while making it in the world there, it’s an amazing experience. And the money doesn’t hurt.
At least if you can handle the negatives. And they’re real. You’re supporting traders. You can be the head of IT, the lead SysAdmin, the brilliant network engineer. Whatever. You’re a monkey who works for the lowest trader. If the company talks about work-life balance, it’s because they’ve had trouble with people quitting recently and that’s what they mentioned. If they don’t, it’s because they don’t believe in it. That said, it’s amazing work. It’s fast work. It’s educating work. It’s inspiring and depressing. If you can disassociate yourself from what you do, and be awesome at doing it, it’s really really worth it. Personally? I have no problems admitting I do what I do for money if the job is interesting enough that I’d do it anyway. Give more to charity if it bothers you. If you can’t handle time time, though, you’re not going to do well. All I can do is try to present it so you can make an educated choice.
Prop trading shops don’t just get free access to all of the exchanges they’re on. Seats are expensive, and for what they’re doing, not really necessary either. They generally get something called Sponsored Access. A big company will sponsor them on the exchange in return for a fee. But it’s not just access to the exchange. If you’re a small trading company, you may have a few dozen million dollars in the real world. So you prove that to, say, Goldman Sachs and they say “Sure, we’ll sponsor you. And so you can trade more, here’s some leverage to play with too!”. Of course, they bill you for how much you trade, so it’s in their interest to let you leverage yourself. So now your measly 8 figures has turned into 10 or more figures of leveraged money.
Look up haircut sometime if you want to know how it all relates to how much you can leverage.
So, your sponsor takes on the risk of you trading somewhat in their name. At the end of the day, they have a list of what you traded, and you have a list of what you traded. The folks in mid-office then have to deal with your clearing firm and your sponsor so everyone agrees just what happened when the traders were out playing all day. The clearing firm is the go-between the two parties of the trade. Basically, they’re a really valuable lubricant in the wheels of the market. So GS gets the list from your clearing firm, you send what you think you traded, and hopefully you don’t get a call from them asking where the 10,000 shares of Berkshire are in your list.
It’s fairly simple actually. You get a method by which you send your report. Let’s say FTP. You upload it with a certain filename format to a server with a given username and password. Easy. The hard part is generating the file. You could have one database, you could have a hundred. There could be one format, or dozens. You could have to download and parse text files. And don’t forget versioning either. So you download all the executions you made. Then you have to group them by exchange, symbol, and side, and add them up and average them out. So your file goes from, say, 200,000 shares of AAPL traded to three lines:
AAPL,B,100203, 190.91,blah,blah AAPL,S,101238,191.33,blah,blah AAPL,SS,2231,191.02,blah,blah
You combine all of them together for that particular exchange, in that particular format, and upload it to your firm. They compare it with their (much simpler to generate, because they only have one potential input) file and if there’s a problem they email and/or call. It’s a surprisingly simple program in perl. Collect your data from all the different sources, put it into a huge hash, print it out to a file, upload it. Put in error checking and notification. Do this for every exchange you trade on, at the specified time(s) daily.
Only two important things to note:
If that file is wrong or doesn’t get uploaded – they don’t trade the next day until it’s fixed.
“The next financial crisis will be caused by a divide-by-zero error in someone’s perl script” (Citation needed)
A huge amount of time and money is spent on compliance at these firms. And at the same time it’s a total afterthought. The traders rule the roost, and they don’t care about it – until they can’t trade anymore or their bonus is lower because of a failed audit. Essentially here’s what it comes down to. You have to keep track of every trade and every order you make, and keep it forever. Sure, they say seven years, or five years, but it’s forever. For one – seven years really is forever in computer terms. Creating something that’s archivable over that long is essentially creating it forever. And for two – there’s nothing saying that a year from now they won’t want the records for 10 years, or 15 years.
So you’re keeping track of every order and every trade. While you’re doing it, you have to keep your positions at the time if at all possible. Why? Shorting stocks for example. Traders are allowed to short, but it goes against the haircut and reduces the amount they can trade. If it’s on the easy-to-borrow list though, it’s a different story. Anyway, the regulators come in and you have to prove that you weren’t shorting a stock. Which means you have to know or be able to derive your position at that time. Then you have to prove that at the millisecond you placed the order, it wouldn’t be a short. Which also means accurate time across the board of your systems. It’s fun, see?
The easy to borrow list is exactly like it sounds. A list of things that are easy to borrow. Some stocks are so available to trade – so liquid – that they get on a special list that doesn’t really count as short selling! Link
In addition to ordering what and when, you also have to be able to prove who. So every trader has to have a unique login so you can prove who did what. If Bob over in the corner is doing something illegal or unethical, you have to be able to prove it was bob doing it. That way when he’s caught he’ll just take himself down and not the company. So you have to record every method by which someone can communicate with evil peoples. You have to keep (like I said, pretty much forever):
Funny story – one time the network went down for some reason. Most places use IP phones now, which use the network. Just to be safe, we had to call the exchanges to cancel our orders – but no phones! So we all whip out our cell phones and call that way. 1- You can’t do that since they’re not logged and monitored. and 2- “Does anyone have something other than AT&T? I can’t call anyone!”
Email has to go through your email servers and be backed up, instant messaging has to go through a proxy that logs all conversation, phones (at least for traders) have to keep recordings, etc etc. One of my better scripts was actually to do a daily dump of chat logs, bundle them per user, and send it as an email to a special email address. Killing two birds with one stone! Regulators will seriously come in and say “we want every communication trader X made from this date to this date.” And you say “okay” or you get a fine. They also do spot-checks at least once per year. ”Give us everything you traded from this date to this date.” And you say “okay” or you get a fine. Incidentally, if you want to see roughly what the code for chat log retention looked like:
#!/usr/bin/perl
use strict;use warnings;
use include;
my $ch='chat_host';
my $cu='chat_user';
my $cp='chat_pass';
my @worries=();
my $to='chatlogs@mycompany.com';
my $today=&include::getdate('YYYY-MM-DD');
my @senders = &include::sql($ch,$cu,$cp,"select distinct(senderid) from message_log where date(modified)='$today'")
or &freakout($!);
foreach my $sender(@senders)
{
my @lines=&include::sql($ch,$cu,$cp,"select msg_text from message_log where date(modified)='$today' and senderid='$sender'");
if(!scalar(@lines))
{
push(@worries,"No lines for $sender");
next;
}
&include::send_email($to,join("_",split(" ",$sender)).'@backuphost.mycompany.com','Chat logs for $today',join("\n",@lines))
or push(@worries,$!);
}
if(scalar(@worries))
{
&include::send_email('me@mycompany.com','chat_backups@backuphost.mycompany.com','Possible problems with chat backups',join("\n",@worries));
}
sub freakout
{
my $wtf=shift;
&include::send_email('me@mycompany.com','chat_backups@backuphost.mycompany.com','Error with chat logs for $today',$wtf);
exit(1);
}
It’s a simplified but only because the hard stuff is hidden in my include module I keep around to make my life easier – it was(is) literally this short. We traded every single day and there was chat (and chatbots) every day, so if there wasn’t anything in the logs there was definitely a problem. In plain language:
Do some housekeeping (includes, variable setup for hostnames and the like, the date, etc) Get a list of everyone who sent an IM that day and if there's a problem, send a freakout email with what happened. For every person in that list, send an email with their name as the "from" address and the chats they sent as the body (if there's a problem, add it to the list of worrying things) If anything is worrying, send an email to that effect to me.
The reason this is so simple is kind of a lucky coincidence. The chat logs are in one table, the ID is the name of the person, and the text of the log includes the time and who it’s to. If it didn’t, I’d have to have another line to get the identity of the person – no big deal. I’d also have to have a temporary array of lines that I’d append the “to”, “time”, and “text” to as one entry, then send the email with the contents of that array instead.
Going to attend the 7th Dutch Perl Workshop tomorrow in Arnhem.
Cannot wait to get up bright and early for a fun and relaxing day dedicated to the fascinating world of Perl. Last year was a blast and with lots of interesting talks I look forward to the trip.
During the ninety minute drive east, I'm going to crank up my latest Porcupine Tree CD and prepare myself for the busy day of mental gymnastics.
http://github.com/tokuhirom/multiperl
"multiperl" is perl5 interpreter installer for perl developers.
% multiperl 5.8.9 -Dusethreads
I dislike "switching" approach, such as rvm.Becase it causes "agghggggggghhh, which perl is not 5.8.1!!!!111111" issue.
And, perl5 hackers don't need switching, I think.Because perl5 hackers don't need to use this tool frequently.
Perl5 has great backward compatibility. perl5++
I really have no energy to write much. The first day of CeBIT was a bit low on visitors but even that went quite well. I met with several people I wanted to talk. Especially interesting were the chats with the people from the Eclipse Foundation.
Today, on the second day, was a lot busier. Many people were walking around and many people came to the Perl stand as well. There are always 3-4 Perl::Staff people at the stand and usually they are busy talking to visitors.
Some PHP and Ruby developers came and we could show them interesting things that can be done with Perl, Moose, Catalyst and DBIx::Class. Many people also liked Padre, you know the Perl IDE
There were many people who said that the main problem they are facing is the lack of Perl developers and some of them told us that they are actually considering switching language because of this. It seems to them that it is easy to find programmers to many other languages, something that was the case with Perl as well a few year ago but now there don't seem to be any Perl programmers. I wonder what could be the reason of that? Where are does developers gone? Have they stopped using Perl and have they remove Perl from their CV? Are there many more jobs now filled by those developers without new developers learning Perl to fill the newly created jobs? Maybe the expectations from a Perl developer are now higher than 5-10 years ago so the same people who could counted as Perl developers 5 years ago aren't considered as such any more?
Anyway we have a lot more work to do in the coming days. I'll try to add some pictures to the next report.
Renee made some nice pictures of Perl on CeBIT
Laufeyjarson asked on Monday, about stripping whitespace from both ends of a string. The comments contains lots of suggestions, but no hard figures, so I thought I'd reproduce them here along with the code used to generate the benchmarks - it provides a simple example of how to write a quick and reliable benchmark.
I've just released v0.20 of DBIx::Log4perl. DBIx::Log4perl is a Perl extension for DBI to selectively log DBI methods, SQL, parameters, result-sets, transactions etc to a Log::Log4perl handle.
This release fixes a problem where execute_array tuple status was reported at INFO level instead of DEBUG level and adds the DBIX_L4P_LOG_DELAYBINDPARAM constant which delays logging bind_param methods in DBI until the execute method is called.
David Golden praised Tatsuhiko Miyagawa's excellent new cpanminus CPAN client in The power of not being all things to all people. You should consider using cpanminus.
Don't overlook something else insightful that David wrote:
It's a lot of work to be all things to all people and I keep wondering whether making things simpler and better for 99% of people would be a better choice.
The only reliable way I've ever seen to "make the easy things easy and the hard things possible" is to make the easy things the default without preventing customization of the hard things. That's a design principle for languages, APIs, and tools.
Figure out what's most common (though not necessarily what people think they want, but what they need). Optimize for that. Consider what they might need and don't prevent it.
That's not easy, and what people need will change over time, but if you want to solve problems well, you have to solve the right problems.
The Perl community seems abuzz about the cool new cpanminus client by Tatsuhiko Miyagawa. After about two weeks of development, this is a reasonably functional CPAN client with just a fraction of the overhead, complexity and verbosity of the CPAN and CPANPLUS clients that come with the Perl core.
via www.dagolden.com
I've been working on yet another CPAN client cpanminus and it's got a lot of attentions and great feedbacks so far. I don't have yet much time to write about it but David Golden who maintains a lot of CPAN ecosystem infrastructure such as Module::Build, Test::Reporter and CPAN.pm, has kindly written a great post reflecting the views from CPAN toolchain people. You must read.
Also, I appreciate your excitement and kind words on cpanminus but please do not just trash CPAN(PLUS). Criticism is fine, but calling them bloated is just too lame. They have reasons for their flexibilities and verbosities to make sure it works everywhere everytime, which cpanminus can fortunately ignore.
The Perl community seems abuzz about the cool new cpanminus client by Tatsuhiko Miyagawa. After about two weeks of development, this is a reasonably functional CPAN client with just a fraction of the overhead, complexity and verbosity of the CPAN and CPANPLUS clients that come with the Perl core.
It’s a remarkable achievement, not only technically, but in the reaction it has sparked. As one of the (sometimes reluctant) maintainers of CPAN and CPANPLUS, I’ve realized that I both love and hate cpanminus.
Miyagawa graciously acknowledges standing on the shoulders of giants. Still, I can’t shake the nagging thought that cpanminus should never have been necessary in the first place.
What I’ve come to realize is that cpanminus is another example of the power of not being all things to all people. Miyagawa doesn’t promise that it works for all of CPAN or that it works everywhere that Perl does. He doesn’t have to. Making it work for 99% of CPAN for 99% of people is more than good enough.
I’ve been co-maintaining various parts of the Perl toolchain for a while now. It’s a frustrating challenge needing to make thing work everywhere, for everything, and trying as hard as possible not to break backwards compatibility. Plus, I don’t even get to use CPAN to make life easier. I don’t get to use handy tools like Moose or DateTime or Regexp::Common or SQLite or anything in the Config::* namespace or even basic tools like Archive::Zip. Nearly everything is done by hand.
Things have to work with just core Perl on a diverse set of platforms and with an incredibly limited set of assumptions. For example, the Perl core still doesn’t come with an HTTP client, so CPAN has to rely on FTP or command line programs to bootstrap LWP. (This is something I personally plan to tackle during the Perl 5.13 development series later this year.)
I think this is an ongoing challenge for core Perl development in general. It’s a lot of work to be all things to all people and I keep wondering whether making things simpler and better for 99% of people would be a better choice. (Anyone else for use strict by default? I hope that finally comes to pass in Perl 5.14.) chromatic writes about this topic often in his Modern Perl blog and I usually tend to agree with the points he makes. (October February 2009 had a particularly good series of posts.)
In the meantime, I look at cpanminus with greed and envy. Miyagawa++
Did you ever wonder how BEGIN, CHECK,
END and so on are called in Perl? Well, they didn't have a good
name, until recently.
The Perl 6 spec listed them under closure traits, which is unwieldy, and not really exact either. Now they are called phasers, because they tell you which execution phase the block or statement runs in.
There are so many possible puns that I'll refrain from writing any.
(This blog post will probably only interest Perl 6 hackers, since it talks only about infrastructure.)
One of the central pieces of Perl 6 infrastructure is the Pugs svn repository. It holds not only the Pugs source code, but also lots of other Perl 6 projects:
It's huge, but at the same time it's very practical: anybody who is interested can get write access very easily, create a new subfolder for a new project, or can fix a typo in someone else's README file without asking for commit access first.
The pugs repo is also viral: Anybody with commit access and invite new committers. Despite what you might think of it: it actually works in practice, so far I haven't seen a single case of abuse.
The pugs repository is hosted on feather, a server kindly provided by Juerd and his company. It contains three virtualized servers, feather{1,2,3}. feather2 is used for "sensitive" data (for example an IRC bot that has API keys for various github accounts, and the perl6.org website). Only a handful of "trusted" users have an account there. feather3 is used for low security stuff like evalbots which might go astray.
feather1 holds all the rest. That means the pugs repository, commitbit (the software we use for handing out commit bits and resetting svn passwords), various websites, and a whole bunch of Perl 6 developers and users have a shell account there, for trying out Perl 6 and hosting their screen + irssi sessions there.
I was about to write feather1 is maintained by a bunch of volunteers, but that would be a lie. It is "maintained" on an as-needed basis by whoever has time and feels half-way competent. It is an "interesting" mixture of Debian unstable and experimental. And it's becoming unmaintainable.
It seems clear what to do: set up a fourth virtual machine, set up a replacement for feather1, and en passant migrate some things (like websites and the pugs repo) to feather2.
But wait! There is an issue. There's always an issue. Setting up a new machine and migrating services takes time. Lots of time. And nobody wants to do it right now. Quite understandably.
Take the pugs repository for example: you might think it's easy enough to
copy /var/svn/... recursively on the new host and set up
Apache... except that you need authentication. And authentication is coupled
to commitbit. And commitbit runs on Jifty. And if you want to install Jifty,
you install half of CPAN. Does anybody know if commitbit is still maintained?
and if it installs cleanly on a Debian stable?
So I thought about some changes to the infrastructure to make it easier to maintain. For example we handle commit bits differently for git projects on github: an IRC bot knows the API keys of the project owners, and can add committers to the projects. That's nice, and the IRC frontend is much leaner than the Jifty-based web frontend. But we lose virality. For security reasons the bot has to keep a whitelist of IRC users who are allowed to invite commiters. Since IRC nick names and (git|svn) user names don't always match, such a list has to be maintained manually.
The second question is: should we take the chance and move the pugs repo to git? I prefer git over svn by far, but there are also costs involved. For example Rakudo checks out a copy of the test suite via svn. The test suite is only a subdirectory of the pugs repo, so unless we split the pugs repo, we'd have to find a way to check out only a part of a git repository. I have no idea if that's possible.
Either way, I'd like to hear your thoughts, and learn from your experience:
Most programming languages handle failures with either of two paradigms: failing routines return special values, or they throw exceptions.
Either way has its severe problems: in languages like C it can be very
simple to forget to catch such a return value, and very tedious to propagate
them to the caller; on the other hand throwing exceptions often clutters the
code with way too many try blocks, and it's generally unfriendly
if you try to automatically parallelize expressions.
So Perl 6 offers a middle ground: soft or unthrown
exceptions. If a routine calls fail("message"), a new
Failure object is created and returned from the current
routine. That object behaves as an undefined value, which stores the message,
file and line information of the fail() location, a backtrace and so on.
When you ask such an object whether it's true or false, or defined or undefined, you'll get a correct answer, and the exception is marked as handled. However if you try to use it as an ordinary value, it turns into an (ordinary) fatal exception. So both of these work:
# Variant 1: no exception thrown my $handle = open('nonexistingfile'); if $handle { .say for $handle.lines; } else { # do something else } # Variant 2 my $handle = open('nonexistingfile'); # throws a fatal exception while calling $handle.lines .say for $handle.lines;
Now if you do some automatically parallelized operations, a single failure doesn't have to abort the whole operation, and neither is information lost
# divide @a1 by @a2 element-wise, a division by zero might occur: @a1 »/« @a2;
The API for accessing the Failure objects isn't very mature
yet, but the concept stands. See S04/Exceptions for the
gory details, as they stand today.
I've blogged about the Perl 6 Advent Calendar 2009, and want to follow up that it's been a huge success so far.
We're about half-way through, and you might be interested in our number of visits per day:
On 6th and 7th of December we had about 18k visitors in sum, courtesy of slashdot and Tim O'Reilly on Twitter.
As a result we got a lot of new people in our #perl6 IRC channel, asking for help on how to get Rakudo working, how to code some specific things in Perl 6, or asking about general design decisions.
Also mj41 reported that he has collected more than 50% more Perl 6/parrot blog posts than in the previous year.
We've also acquired
perl6.org for our uses this year, and it has been promoted enough to be
the third hit on a google search for Perl 6.
All in all I'm rather happy with the marketing state of Perl 6 in 2009, and hope to see similar efforts for 2010. With the upcoming releases of Rakudo Star and a Perl 6 book I'm pretty sure we'll do well, and I hope for more slashdot coverage :-).
These tidings posts seem to become rather sparse, but I hope you get some news by reading the Planet Six feed aggregator anyway.
for 1, 3 ... *+2, 9 { .say } to print all
the odd numbers between 1 and 9. (r28344,
r28348,
r28351).Rational and Complex types now have their own literals
(r28173).As opposed to two months ago, Rakudo now
Patrick Michaud is also working on a new tool named npq-rx, which combines a self-hosting NQP compiler and a new regex engine, which already supports proto regexes, NQP code assertions and closures, and is generally much faster and better than PGE.
Mathew Wilson aka diakopter started sprixel, a Perl 6 to Javascript compiler.
Mildew now has an experimental javascript emitter.
perl6.org is redesigned again, this time spanning multiple pages, thus allowing much more stuff to be linked there.
Four Perl 6 and Rakudo hackers announced that they are writing a Perl 6 book, the print release of which shall coincide with the release of Rakudo Star.
I think it's not too bold to call the Perl 6 Advent Calendar a full success: over 40k page views, more than 150 non-spam comments, and lots of new faces and nicknames in #perl6 - it's been a very pleasant surprise.
I asked myself why it became such a success, and came up with this list of things:
24 days are over rather quickly, so everyone who contributes knows when the deed is done
Some people like the shiny new regex and grammar features, others are interested in the object model, operator overloading or complex numbers. Since we had no fixed topics, everybody could choose a topic that played to their strengths.
Writing a good post about a topic you know takes about half an hour up to an hour, maybe two if you need to do some research on the topic. Either way it's a relatively small task (compared to "write a regex engine" or "redesign the build system of $compiler" or so).
We had a schedule for each day, and people could add their name to a free slot. When the day approached, the other authors started to ask politely how it was progressing, making sure people did not forget their posts. Also authors felt the subtle pressure to finish their posts on time.
We had lots of feedback, most of it quite positive: blog comments, visitors in our IRC channel, being featured on slashdot. That was very encouraging.
Also other authors would preview and proof-read the posts before they were published, pointing out falsities and gems.
PerlJam and colomon took care. The decided to make the advent calendar happen, set up a blog, discussed things and so on - they made it happen.
All these factors encouraged contributions. I don't think all of them are necessary for a successful, lively projects, but they certainly help.
What other factors do you know that encourage contribution in open-source projects? What could we have done even better?
We want a Perl 6 book. We want it badly enough to write it ourselves. So that's what we're doing: writing one.
We, that is Patrick Michaud (architect of the Rakudo Perl compiler), Jonathan Worthington (prolific contributor to both Rakudo and Parrot), Carl Mäsak (frenetic Rakudo user, and our number one bug finder) and Moritz Lenz (keeper of the Perl 6 test suite, and Perl 6 user and blogger). We are also open to contribution from others - already Jonathan Scott Duff has written an initial preface for us.
We don't have a name yet for our book. We want to cover the basics of Perl 6, enough to get your feet wet, and enough to make you want to use it. We want it to be based on useful examples. It is not going to be the definitive book, that task we leave to Larry Wall and Damian Conway.
Our vision is to present primarily the subset of Perl 6 that Rakudo understands, and have printed copies available by the time Rakudo Star is released, that is April or May 2010. chromatic and Allison Randal have kindly offered to published it via Onyx Neon Press.
Until then, monthly releases will be published under a Creative Commons license (noncommercial, attribution, share-alike).
Currently we have four chapters under construction, and the intention of writing the more introductory chapters later, when we know what we need to introduce for the later chapters. So far we have
You can download the preliminary PDF version of the book here.
Interested? Check out
the git repository, and
join us in irc://freenode.net#perl6book.
In the great tradition of Perl Advent Calendars, colomon started and announced the 2009 Perl 6 Advent Calendar, with a post about Perl 6 each day.
After the first post many #perl6 regulars volunteered to contribute a post, so 20 of the 24 days are already allocated.
I'm looking forward to many nice posts, most of which will probably highlight a small Perl 6 feature.
In Perl 5 there is the undef value. Uninitialized variables
contain undef, as well as non-existing hash values, reading from
unopened or exhausted file handles and so on.
In Perl 6 the situation is a bit more complicated: variables can have a type constraint, and are initialized with the corresponding type object:
my Int $x; say Int.WHAT(); # Int()
These type objects are also undefined, but in Perl 6 that doesn't
mean they are a magical value named undef, but that they respond
with False to the defined() subroutine and
method.
In fact there is no undef anymore. Instead there are various
values that can take its place:
Mu is the type object of the root type of the object hierarchy
(or put differently, every object in Perl 6 conforms to Mu). It's
the most general undefined value you can think of.
Nil is a "magic" value: in item (scalar) context it evaluates to
Mu, in list context it evaluates to the empty list. It's the
nothing to see here, move along value.
Each type has a type object; if you want to return a string, but can't
decide which, just return a Str.
Other interesting undefined values are Exception (which
usually contain a message and a back trace), Failure (unthrown
exceptions), Whatever is a generic placeholder that can stand for
"all", "infinitely many", "many" or as a placeholder for a real value.
A few days ago masak blogged about the social aspect of the is Perl 6 really Perl? question.
He presumes that the answer is yes, but doesn't tell us why. I'll try to give some reasons.
Perl 6 started off as the successor to Perl 5, at a Perl 5 meeting, by the Perl crowd. It was a plan to escape both the backwards compatibility trap (which meant that broken things couldn't be fixed without many people yelling), and the lack of momentum in the community.
What makes Perl Perl? In my opinion it's not the sigils on variables that make Perl Perl, or that writing a regex only need two characters and so on. It's mostly the philosophy that makes the difference.
There are some underlying principles like TIMTOWTDI, context sensitivity, convenience over consistency, making simple things easy and hard things possible, often used constructs short and less frequent things longer, and so on.
Perl 6 is founded on all those philosophies and ideals, and also shares
some technical principles. For example sigils on variables (oh, I mentioned
them already ...), easy access to powerful regexes, that fact that operations
coerce their arguments (instead of the type of arguments determining the
operation like in javascript, where a + can either mean addition
or string concatenation).
So if you agree with my definition of what makes Perl Perl, Perl 6 is also Perl. If not, please tell me what's the essence of Perl!
When you learn a programming language, you not only learn about the syntax, semantics and core libraries, but also the coding style and common idioms.
Idioms are common usage patterns; learning and reusing them means you have to spend less time thinking on common things, and have more time working out the algorithms you deal with.
That's different if you learn Perl 6 - it's a largely unexplored field, and while there are loads of nice features, you might still feel a bit lost. At least I do. That's because I often think "There's got to be a much easier way to achieve $this, but it often takes time to find that easier solution - because nobody developed an idiom for it.
In those cases it helps to ask on the #perl6 IRC channel; many smart people read and write there, and are rather good in coming up with simpler solutions.
For example see masak's ROT13 implementation in Perl 6. In the right column you can see later revisions, and how they gradually improve, steady up to a one-liner.
I also made some simplifications to JSON::Tiny, which basically shows that when I wrote these reduction methods first I used Perl 6 baby talk language.
The nice things about exploring the Perl 6 wonderland of unexplored idioms is that it really pushes your ego if you find a nice simplification, and that you have something to blog about for the Planet Perl Iron man ;-)
You can find it here.
This version should enable GnuCash users to get quotes from websites again. Other work went to integrating HU.pm (Hungarian stocks) and bug fixes due to website modifications.
Finance::Quote is known to work nicely with Finance::QuoteDB for keeping your quotes easily in a database. Finance::QuoteDB uses DBIx::Class and therefor it is database independent. A frontend script (fqdb) is available if you just want to start downloading stocks to a personal database. Historical quotes are also available through the use of Finance::QuoteHist which is also supported in Finance::QuoteDB. An interface to GeniusTrader is also available.
Futur work goes to the porting of GeniusTrader to CPAN. A alpha version has already been posted to CPAN but non-developpers should stick to the current stable release of GeniusTrader