entries
entries
First off, I will note that I've decided to make a little Amazon "aStore" for stuff I find myself recommending a lot. Be forewarned that I'm going to link to it when I link to stuff I endorse, and that if you buy from it, I get some cash. If you find this repellant, copy the ASIN and search for it at Amazon and buy there.
My father recently had some major heart surgery. (He's fine now, and the speed of his recovery is pretty impressive!) He got stuck in the hospital for about a week, and I told him he could borrow my PRS-300 to read stuff while stuck in bed for all that time. Unfortunately, it didn't quite work for him, and I was off at OSCON. When I got back, I rebooted it and it was fine, but he didn't get any quality time with it, so I loaned it back to him. After a day or so with it, he seemed sort of semi-interested. He was also kind of interested for me to bring by my other, bigger ebook. I was surprised, because I think of it as a pretty special-purpose device. Then again, my dad is a serious gadget man -- much moreso than I am.
So, I went back the next day with my beloved Kindle DX. I use mine for reading technical books, because they just don't scale down nicely to the little five inch screen on the Sony reader. Dad was captivated. He could read at all kinds of font sizes -- the Sony supports three, but anything small is often pointless, since you only get a few words on the screen at once otherwise. He could print genealogy reports as PDFs and show them to relatives while visiting. He could take them with him to rural churches where researching more family history, for reference. He liked the text-to-speech feature. Shortly after I left, he went ahead and ordered it. I'm really interested to see how he likes it, after some time. I'm even more interested to see whether it can be used as leverage to get him to donate away some of his huge collection of unlikely-to-be-read-again books.
As for me, like I said, I really like my DX. My big problem is getting books for it. Many of the best classic and important texts on programming are just entirely unavailable on ebook. Addison-Wesley and Pearson titles are the worst offenders, possibly because they published some of the most important early books on UNIX and C programming. There don't seem to be legal editions of Software Tools, The Unix Programming Environment, TCP/IP Illustrated, Advanced Programming in the UNIX Environment, or dozens of other really, really great, important titles. It's not just that I want to have digital copies for fun, either. APUE, for example, is almost a thousand pages long, 2.5 inches thick, and weighs almost five pounds. This is not a book I can stick in my bag to carry around and read! The same goes for plenty of other really useful technical books. I am against buying books with restrictions on their use, but to make these books usable, I'd probably suffer through it. I could really use the shelf space, too.
O'Reilly Media seems to be way ahead of everyone else in the field. Not only do they offer a large fraction of their catalog in electronic form, but they tend to do it in multiple formats (epub, mobi, PDF, and more), and they do it without DRM. They also have a "deal of the day" that drops the price of one ebook to $10. It's usually something I'm not interested in, but once or twice a month at least I buy the book and get a great deal on something. I was also very excited when I saw that their Safari Bookshelf program seemed to be targeting Kindle for its mobile interface. Unfortunately, what I found was that it was hard to navigate, unpleasant to use, slow, and often just inaccessible when I was on the road. This is where O'Reilly should embrace some kind of scheme for limited duration book checkout, as some libraries now do with EPUB books. I don't think Amazon provides a way to do this, but it would add a lot of value to Safari Bookshelf and Kindle. I was so disappointed in the performance of Safari Bookshelf on Kindle that I ended up canceling my account. I'd definitely go back if I could get books onto my reader.
When I want to read web pages on the Kindle, I tend to go through Instapaper, which can strip articles like news sites, Wikipedia pages, or blog entries down to just formatted text. I've set it up to send my unread articles to me wirelessly so I can read them offline. They show up looking something like magazine, and they're much easier to read offline than when sitting at a browser with lots of other windows trying to distract me.
Now that I'm reading more and more on my Kindle DX, I'm starting to find the things I really do and don't like. The basic likes are pretty easy to imagine: it's light, it holds a charge for days or weeks, it holds all my technical ebooks at once, it's easy to get books onto, and when absolutely necessary it makes a tolerable Wikipedia reader. The fact that it can read full-page PDFs without scaling down to an unreadable or squint-inducing scale is also a great thing, and a large part of what makes the DX so useful for me -- I can read technical books and scanned journal papers easily.
Unfortunately, every once in a while an ebook comes along that is just no good even on the DX. The original free version of Higher-Order Perl was one of these, because every page had huge margins that did nothing but cause the page to be zoomed out until illegible. (Joe McMahon very kindly removed this whitespace and the HOP website now offers a much Kindle-DX-friendlier PDF.) O'Reilly's "Rough Cuts," seem to have this problem, too. They only offer their books in PDF, which is fine, but they tack on large margins and footers that cause the whole page to become nearly unreadable.
Surely, the Kindle DX can zoom in and pan around the page, but this is slow, ugly, and basically not something that anyone would really want to do to read a book. It's nice that the feature was added, but it really doesn't mitigate the "I can't read this PDF because of the font size" problem. Or, rather, it replaces that problem with one of nearly equal obnoxiousness.
PDFs are, in most ways, inferior for reading on the Kindle DX. They can't be reasonably font-resized because they're already typeset. (This isn't Kindle's fault, it's just the way things are.) They aren't as searchable. Their metadata isn't as easy to edit. They don't offer as many "jump to" locations -- except for one kind: pages. PDFs, because they are typeset, have pages. Ebook formats, on the Kindle, do not. This drives me batty. I realize, of course, that there are no physical pages, but including page number markers would be really welcome. The EPUB format supports this. On my Sony, the current position indicator tells me that I'm on page 4, or seeing parts of both page 4 and 5. Sure, at most font sizes I will be "on page 4" for several page turns, but that's okay. On Kindle, most of my books seem to have thousands of "locations," which are presumably something like "paragraphs." I haven't looked into it. This is a problem when someone asks, "What page is that on?" I have no idea. "It's at location 48,102." doesn't help them. I can't even tell them what chapter or section it's in, either, because Kindle ebooks do not usually have headers or footers announcing the current section, which a PDF book does show.
On a related note, it's nice that page breaks no longer need to be a hassle. If a code listing seems to be broken at a page turn, I can advance just half a screen's worth of lines to get the code listing centered on one screen. It's unfortunate that to do so, I need to manually enter a new location number, instead of just hitting something like Shift+NextPage.
As I put more books onto my Kindle, I'm finding it harder to get to just the book I want. The Kindle book-browsing interface is no iTunes. They did recently add "Collections," but they can only be managed from the device itself, and that's pretty slow and gross. It would be nice to get an OS X program that could organize my Kindle's content for me by manipulating the files on it directly. Instead, I'm stuck with Calibre, a program that is both fantastic and horrible at the same time.
Finally, there are all the little things that bug me about Kindle: I wish I could change the sleep screen images without "hacking" the software; I wish bookmarks and notes were easier to add. Kindle recently added a Twitter client for posting excerpts, but it can't post random tweets, and posts from my own PDFs only credit "personal document" and not the book by name.
In the end, though, the Kindle DX has been well worth the price, and I'm definitely getting a lot more technical reading done. I'm hoping that the future brings us a Kindle more on the scale of the Sony PRS-300, because even the Kindle 3 is just a bit too big for my pockets.
Last week, I was at OSCON. I have to write a lot more about that whole experience. For now, it's only relevant because it mean that when my Popbox arrived, I wasn't home. I checked some of the reviews and they were bad. Like, shockingly bad.
When I arrived home, Popbox had issued a few firmware updates, so I remained hopeful. Unfortunately, they utterly bricked the box. I sent it back and ordered an Acer Aspire Revo to use as an Ubuntu machine with XBMC. It showed up today.
It was pretty frustrating as attempt after attempt failed to get the darned thing working. I just finished watching an episode of Batman: The Animated Series, though, so I'm hoping that everything is now right with the world.
First, I couldn't get the XBMCLive system installed. I could get it onto a USB stick, but that would boot to a login prompt, not the XBMC system with an installer. I tried this a few times, and gave up in disgust. Then I tried to install it to a hard drive, but grub died. The drive changed its device name between setup and attempted install, but grub was dying too early for me to fix the setup.
Finally I installed Ubuntu 10.04 to a micro SD card and installed Ubuntu from
there. After that, I let it do all of its upgrades, installed
xbmc-standalone, and set it up to run that automatically on boot. This was
progress!
Next up, it couldn't find my RaLink RT3090 wireless chip. The guys in
#xbmc-linux were alternately helpful and not, but they kept my brain active
while I kept looking. Finally, I found this bizarre
solution
in a forum:
As root:
mkdir -p /etc/Wireless/RT2860STA/
touch /etc/Wireless/RT2860STA/RT2860STA.dat
service network-manager restart
...and that's it. Wuh? Well, it worked.
So, I took the thing downstairs and hooked it up. Everything seemed okay,
until I tried to play my first video. It took ages to buffer anything and
playback was unacceptable. I'd gotten some grief from IRC about using wireless
for streaming video, but I do this all the time already to my Xbox 360 and PS3.
Further, I wasn't getting any audio over HDMI. I struggled with that for a
while before finding another useful forum
post
suggesting that the HDMI output was muted, and that the only way to find it was
to run the terminal alsamixer program and hit the undocumented "m for mute"
command. That did the trick!
After that, I tried to play an episode of Batman, and it worked. Why was the wireless weird before? No idea. It would be nice to get a cable run, somehow, but I am in no hurry.
I also installed XBMC Remote on my iPhone, but I think it's sort of underwhelming. I'm not sure what I'll do in its stead, but I don't think it's a long term solution. I might just get an IR receiver and set up the XBMC in our Harmony remote.
My hope is that we can use this device to let Martha see whatever of her DVDs she wants without having to fumble with discs. Any other application is pure gravy.
I just needed to create a Yahoo! account for someone who wanted to be able to access some of my photos. This required creating security questions and answers, and as usual the questions available to pick from were really awful. It just got stupider, though, when I tried to pick answers that I would be able to remember. (Many security questions are horrible because their answers change over time.)
Q: What is your youngest child's nickname?
A: Rik
This was disallowed, because the nickname must be at least four characters. What??
Q: What is the name of your favorite uncle?
A: Uncle
Disallowed: the answer cannot contain part of the question.
Q: Where did you get engaged?
A: Earth
Q: Where did you spend your summers as a child?
A: Earth
Disallowed: you can't have the same answer for both questions.
How do these policies help anyone?
On top of this, the last time I needed to recover an old Yahoo! account that predated security questions, the help staff closed my ticket basically saying, "You need to answer the security questions."
Thanks, Yahoo! You're just as helpful as you'd be if Microsoft had bought you!
I'm finally on the last leg of my trip home from YAPC. The travel home was slightly chaotic, and I worked a full day before actually getting onto the road home, but it's been fine. The planes were fine, I was seated next to Walt (by chance!) and I didn't suffer any extreme delays. That might change, as it looks like I'm going to be over an hour late getting home, but maybe that won't come to pass. It would be nice to get to see Martha awake before tomorrow morning. Still, Gloria will be awake, and hopefully we can relax and maybe catch up on Top Chef a little.
The trip was good. The venue and housing were good, there was a good selection of talks, there was a lot of good food in the area, and I got to see almost everyone I had expected to. Every year I find it harder to summarize what I did, because more of it was conversation and less was attending talks with specific technical deliverables.
I had a number of good talks about API design, Perl language feature wishlists, English orthography, liquor, metaprogramming, and other topics technical and otherwise. I ate a pizza with caramelized onions, gorgonzola, pecans, and dried sour cherries. It was great. I had four local beers, and they were all good. I spent a lot of the conference with a wicked headache, and it helped send me to bed at reasonable times, which was probably for the best.
I gave three talks: one on Git, one on
Dist::Zilla, and one on Perl
5.12. I think they all went
fairly well, although I had mixed feelings about each of them. I'll be giving
my Dist::Zilla talk at
OSCON later this
year, and I think I can make some significant improvements. (One of these will
just be talking about the newly-added dzil setup command.)
I wanted to use my time away from work to get some of my backlogged personal projects worked on. I tried to get some serious progress made on Rx, and I did, but there were a number of steps backwards as well as forward. (Also, Rx is something that will definitely help with word more, so I wasn't even entirely avoiding work work.) I updated all the non-Perl implementations to consume the new-style test suite that I'd put together, but then realized that the error model was really awful and redid it. Then I realized that updating each implementation yet again was going to be a nightmare, so I wrote a test suite generator. It generates a very simple spec test datafile from my much more human-friendly input files. Now, of course, I need to update everything to use that. It will be a bunch of work, but mostly deleting.
The big pain is going to be implementing the new error trees in a reusable way that I can manage in more or less every language I want. Hopefully I'll get Perl done soon, and then everything but PHP will follow shortly thereafter. I might try to cajole someone else to do PHP for me, so I can avoid it a bit longer.
When I got fed up with Data::Rx work, I decided to get through a few random bits of work I had lined up. I integrated a number of changes to Git-Megapull from Ævar and wrote my stupid little "cherry pick a whole remote branch" script for dealing with multiple contributed feature branches. Then, despite my initial plan to take a break from Dist::Zilla, I did a bit of work on that, too. I released some changes that had been languishing for a few days and added the input methods for Chrome. This is something I've been waffling over for a long time, because all terminal input libraries are horrible. I finally settled with Term::UI and plunged forward. I think this will help lessen the new user's confusion at least a bit.
Tonight, I'm going to relax with Gloria and possibly set up my new phone. This weekend, I will play with my daughter and go to my nephew's birthday party. Pretty soon, though, it will be time to really get cracking on my Moose is Perl talk at OSCON, which is on stage in about three weeks. I'll probably also continue my troubled efforts to document Dist::Zilla more fully, both in the tutorial and its shipped Pod. Oh, and I need to work on recording audio for a slidecast of my Dist::Zilla talk.
I've just finished, but not released, a dzil setup command. If a user runs dzil (for most purposes) without a config.ini present, this warning is displayed:
~/code/cs/Dist-Zilla$ dzil build
WARNING: No global configuration file was found in ~/.dzil -- this limits the
ability of Dist::Zilla to perform some tasks. You can run "dzil setup" to
create a simple first-pass configuration file, or you can touch the file
~/.dzil/config.ini to suppress this message in the future.
If you run dzil setup, you see this:
~/code/cs/Dist-Zilla$ dzil setup
What's your name? Ricardo Signes
What's your email address? rjbs@cpan.org
Who, by default, holds the copyright on your code? [Ricardo Signes]:
What license will you use by default (Perl_5, BSD, etc.)? [Perl_5]:
Do you want to enter your PAUSE account details? [y/N]: y
What is your PAUSE id? [RJBS]:
What is your PAUSE password? PeasAreDelicious
config.ini file created!
...and after doing that, you have a ~/.dzil/config.ini:
[%User]
name = Ricardo Signes
email = rjbs@cpan.org
[%Rights]
license_class = Perl_5
copyright_holder = Ricardo Signes
[%PAUSE]
username = RJBS
password = PeasAreDelicious
Hopefully this will make it a bit easier to get started directly out of the box.
Spoilers may follow.
Last night, I finished reading the big DC crossover event, Blackest Night. (That link goes to the collection of the eight core issues, not the full hundred issues of crossover matieral.) I'd read all the preliminary material, but had stopped at "Prelude to Blackest Night" (or whatever it was called) so that I could wait until I had all the issues to read them in one go. With all 100 (or so) issues in my possession, I got to work this week and tore through it. It was good, but it had a bunch of problems.
It was too long, for one thing. By the end, I was sick of it. There were so many specific fights that were basically all the same as each other, with nearly nothing of interest unless you really wanted to see how Blackest Night affected, say, Captain Cold. They didn't just do this to show how Blackest Night was everywhere, of course, they did it so they could use the big crossover event as a massive change of various characters. Hawk came back, Captain Boomerang, Jr. died, Zoom is active again, whatever. I'd prefer if DC didn't need to have a huge crisis event every five years to produce sweeping changes.
They did a lot of buildup to the White Lanterns, but it seemed totally wasted, in the end. Sinestro as White Lantern was a joke. They forget that he really was the greatest Green Lantern, sometimes, and make him act like a second-rate villain. If Jordan is greater than Sinestro, it shouldn't be by all that much. Otherwise, who cares? The WL corp should have been introduced much earlier, so we could've seen them do something beyond one boring collective action.
I will not get into all the stupid plot holes or continuity errors. There were some, and I wish there had been fewer.
They acted like Black Hand was going to be a big deal in Blackest Night. He was important, in theory, but he really only got about two or three issues of appearance and after that he was, at best, an occasional Renfield-shaped shadow in the vicinity of (yawn) Nekron. And where was Black Flash? He totally belonged here as more than just an offhand allusion.
The Superboy Prime stuff was horrible. It was some of the worst self-referential meta-crap that I've read from DC in a long time, and writing that sort of drivel that jokes about how everyone know it's drivel does not make it any less drivelly! Similarly, it seemed clear that Vibe was brought back only as a sly wink at long-time readers. It was a waste. The old Doctor Light was also underwhelming, considering his importance in Identity Crisis. With a huge army of people who play on emotional ties, not using Doctor Light to the fullest seemed like either a case of amnesia or playing it overly safe.
The scale of the event was really hard to fathom. Did billions of zombies really descent on Coast City? Really? How long did Blackest Night last for? I'm not sure it wasn't just "one night," given that I can't remember a single shot of Black Lanterns in daytime, but it was really very unclear. Were there massive normal-human casualties? It seems likely, but it was never addressed. Typical! In all hundred issues, I only remember one shot of normal people reacting to everything. It was Hal Jordan's brother's family looking out of their apartment window at the battle and trying not to be afraid. That's it. Nothing else. Maybe every major DC crisis-like event should be followed by a Marvels-like mini showing us what happened to the 4% of humans without super powers.
Even so, I'd probably call this the best (least frustrating) major DC crossover. Just hope that they can stop doing these huge crossovers for a while again. I also won't mind if they keep the other lanterns around, as long as I don't need to see every other DC character briefly become a member of each corps in turn. Even seeing Guy Gardner as a Red Lantern wasn't worth it. The only two temporary conversions I thought were worth it were Wonder Woman as a Violet Lantern and Barry Allen as a Blue Lantern.
We'll see what happens.
YAPC is only about three weeks away, and I have a lot of work to do on my presentation material. I'm giving three presentations: one on Perl 5.12, one on Git, and one on Dist::Zilla. The Dist::Zilla talk will also be presented at OSCON. I've been working feverishly to get prepared for those talks, but not by writing presentation material.
Instead, I've been working on Dist::Zilla itself. I want to make a lot of
improvements to dzil new before I give the presentation, and that means
finishing a lot of long-standing work on the internal configuration system.
Once that refactoring is done, I can add a proper interface to "global
configuration" for config shared between projects. Then there will still be
lots more room to improve the config system, but more importantly I can make
dzil new really easy to use.
Getting these changes into Dist::Zilla (which will be Dist::Zilla v4 when this set of changes is landed) has been a significant amount of work, not just to Dist::Zilla but also to Config::MVP, its external configuration loading system. I'm very happy with the last few changesets, which introduced finalization, temporary references back up from sections to sequences to assemblers, and an overhauled API for writing configuration readers.
Because of these changes, the global-config branch of Dist::Zilla has
replaced a bunch of ugly code with much saner and simpler code. It also can
register each plugin as its configuration section is read, rather than loading
all config, then registering all plugins. The next step is to have non-Plugin
configuration entries, primarily for storing data only. A section, for
example, could store your PAUSE credentials. You might store them in a global
(to your user) file in ~/.dzil, but override that section on a per-project
basis in dist.ini in your dist.
Unfortunately, all these changes have made documentation wrong. When I have noticed the incorrect documentation (and I think I've noticed just about all of it as I worked), I deleted it. So, now there's less documentation of Config::MVP again. Fortunately, I'll have plenty of time to redocument it when I'm preparing a presentation on Dist::Zilla! My current plan, though it may change, is to use most of my "free time" at YAPC and OSCON writing docs rather than code. We'll see how that goes.
At any rate, I'm very happy with the progress toward an improved global configuration system, especially because I know the milestones that will appear next.
I wrote about Soulver about three years ago when I accidentally acquired a free license. This morning, I saw Marcus promoting a new version and I thought I'd give it a look.
On one hand, it looks to have gotten lots of neat new features, some of which could be very useful. It also no longer bills itself as understanding mathematical expressions in English quite so literally. It focuses more on simple, loosely structured shorthand for simple math. On the other hand, it still accepts lots of stuff that it just doesn't know what to do with. It doesn't say, "I have no idea what I'm doing, so I can't answer." Instead, it just does crazy things.
Here's the same set of questions I posed it three years ago, with the old and new answers.
Q: 2 to the power of 2
O: 4
N: 4
?: right!
Q: 2 squared
O: 2
N: 2
?: wrong!
Q: square root of 2
O: 1.414...
N: 2
?: was right, now wrong
Q: cube root of 2
O: 1.414...
N: 2
?: wrong!
Q: average of 1, 2, and 3
O: 6
N: 5
?: wrong!
Q: sum of 1, 2, and 3
O: 6
N: 5
?: was right, now wrong
Q: 1, 2, and 3
O: 6
N: 15
?: was right, now wrong
Q: quotient of 9 and 3
O: 12
N: 12
?: wrong!
Q: minimum of 1 or 3
O: 13
N: 3
?: was mind-bogglingly wrong, now just wrong
Q: 1 or 2 or 3
O: 123
N: 6
?: was wtf, now just wrong
There are some minor improvements to mitigate these problems. Syntax highlighting seems like it helps you spot what things Soulver understood and what things it's just guessing at, but it's still a real mess. See how "2 to the power of 2" was right? "of" is just being interpreted as multiplication, so "2 to the power of 9" is 18. Despite this, the "of" doesn't highlight the way it does if you say "2 of 9"
In most of their demo documents, everything can be parsed. The only weird stuff is units, where you can say something like "2 breakfasts at $20" and get $40. If units were made a special case, then I bet almost all unknown crap could be rejected and this would work much more like what you expected.
Perhaps even more usefully, a column could be added to show a canonical representation of what you tried to say. So you'd enter "1, 2, and 3" and this column would display "1 + 2 + 3". I'd especially like this because it would let me understand why "1 or 2 or 3" is 6. (I think it's because any unknown token between two numbers is interpreted as addition, but I'm not sure.)
The following is a message I recently sent to the Dist::Zilla mailing list:
Config::MVP is the mass of moving parts behind the config loader that does most of the work turning your dist.ini into a set of plugins. For months, now, I have had a good idea about how Config::MVP needs to be extended to make a lot of significant improvements to Dist::Zilla possible.
Today, I've implemented most of those changes, and I will probably implement more soon. After that, I can begin making use of the new features in Dist::Zilla, which will probably get a v3 release in the coming weeks.
Config::MVP has three main parts: Sections, which are (name, associated package, hashref payload); Sequences, which are ordered lists of Sections with unie names; and the Assembler, which is a simple state machine used to build a Sequence. The Sequence is the "final product" that represents the loaded config.
So far, no MVP object required an object above it. That is, you could have a Section outside a Sequence and a Sequence with no Assembler. This has changed somewhat in my master branch, as of now. Now, all three objects can be marked "finalized," and a Section cannot be finalized unless it was placed into a Sequence. More importantly, a Section has a reference to the sequence it was placed in, and a Sequence has a reference to the Assembler constructing it.
This is important because it makes the following possible:
package DZilSection;
use Moose;
extends 'Config::MVP::Section';
after finalize => sub {
my ($self) = @_;
$self->sequence->assembler->zilla->register_plugin($self);
};
That's just a sketch, but I hope it's clear: plugins will be able to self-register immediately during loading, and if necessary perform other operations.
Also, it will make the following vital configuration possible:
[AwesomePlugin]
:version = 1.23
...which will cause something like this:
$seq->section_named('AwesomePlugin')->package->VERSION(1.23);
There is another implication of the above code: the Dist::Zilla object can (must!) exist before configuration is done being read. This means all core attributes will become, in one way or another, deferrable and settable via plugin.
Non-phase-plugin sections will also be able to do things. For example, sections will exist that represent just hunks of data, possibly validated or made into objects, but not contributing to the build-like operations. These will get their own lookup, with fallbacks to global configuration. So, you can say:
$zilla->config_section('Foo')->...
...and you'll get the Foo section from your dist.ini, and if that's not there,
you'll get the one from global config -- falling back to either nothing or a
master default, like the default :Plugins currently supplied. This mechanism
will be used for things like PAUSE credentials and default copyright/license
data for minting new dists, making profiles much more reusable by multiple
users. (It'll also finally mean that dzil new can be documented without
looking half-baked.)
Finally, bundles may change. Rather than immediate transformation into sections, bundle processing will probably end up processing each Bundle as its own section, complete with finalization, and a side effect of that finalization will be the unshifting of synthetic input (the bundle_config) onto a to-be-created MVP config queue. The biggest benefit of this change is the ability to inspect the Dist::Zilla object and report what bundles were actually loaded and in what order. This will help the MetaConfig plugin paint a more complete picture.
That's my whole update for the night, but I'm very excited about these changes, and I hope you will be, too.
I'm pretty darn cautious about upgrading stuff on servers, but for some reason
every once in a while I lose my mind and say, "I'm going to do a dist-upgrade
my primary workstation right before (bedtime, work, a deadline)." I did that
last night.
Everything was fine, although I had to re-install a bunch of Perl libraries.
(I should be using local::lib or perlbrew more often.) Then I ran mutt and
I got prompted for my password... even though it was connecting via a direct,
preauthenticated tunnel.
It took me a long time to feel certain that it was this: When running imapd,
I got the following:
* OK [ALERT] Filesystem notification initialization error -- contact your mail administrator (check for configuration errors with the FAM/Gamin library)
* PREAUTH Ready.
This should not have been an issue, as I understand IMAP -- but my understanding isn't great. It's a weird protocol. I looked and looked for how to disable the advanced IMAP idle features that needed the file monitors, but as far as I could tell, Courier would always look for FAM or Gamin. Finally, I threw up my arms in frustration and installed Gamin.
Everything just worked!
Last night, I was making some improvements to some work code, and trying to make it easier to use in places where I wasn't using it yet. Part of this involved allowing an important attibute to be a weak reference. I really wanted to do something like this:
my $child = $parent->child;
my $parent = $child->parent;
In this code, $parent and $child may be expensive to create, so I want both
to have a reference to each other, without worrying about cyclical references,
which will prevent either variable from ever being garbage collected. That
means one of the references has to be weak. I picked the reference from
$child to $parent. Unfortunately, this means that $parent can be garbage
collected before $child, which can lead to this problem:
my $parent = Parent->new(...);
my $child = $parent->child;
$child->parent; # returns $parent
undef $parent;
$child->parent; # returns undef, which may violate the attribute's type
Alternately, here's a runnable example that demonstrates just the problematic behavior.
use 5.12.0;
our $parent;
package Child;
use Moose;
has parent => (
is => 'ro',
isa => 'Ref',
lazy => 1,
default => sub {
$parent = {};
return $parent;
},
weak_ref => 1,
);
package main;
my $child = Child->new;
use Test::More;
is($parent, undef, '$parent begins undef');
is_deeply($child->parent, {}, '$child->parent returns {}');
is_deeply($parent, {}, 'now $parent is set');
undef $parent;
is($parent, undef, '$parent has been undefined');
is_deeply($child->parent, {}, '$child->parent returns {}');
done_testing;
I ended up hiding the reader method for the parent variable and replacing it
with one that checks the definedness of the attribute. If it's not defined, it
clears the attribute and re-accesses it, which causes the lazy constructor to
fire again.
has parent => (
is => 'bare',
reader => '_parent',
isa => 'Parent',
lazy => 1,
weak_ref => 1,
clearer => '_clear_parent',
default => sub {
my ($self) = @_;
Parent->get_parent_of($self);
},
);
sub parent {
my ($self) = @_;
my $value = $self->_parent
return $value if defined $value;
$self->_clear_parent;
return $self->_parent;
}
This is already pretty complicated, but it's still not enough. When the value
in $parent gets garbage collected, the next time we try to read it, our
method notices that it's undef and gets a new value. Unfortunately, because
nothing else refers to the new value it is immediately garbage collected. We
need to only weaken references that were set during construction.
has parent => (
is => 'bare',
reader => '_parent',
isa => 'Parent',
lazy => 1,
predicate => '_has_parent',
clearer => '_clear_parent',
default => sub {
my ($self) = @_;
Parent->get_parent_for($self);
},
);
sub parent {
my ($self) = @_;
my $value = $self->_parent
return $value if defined $value;
$self->_clear_parent;
return $self->_parent;
}
sub BUILD {
my ($self) = @_;
Scalar::Util::weaken $self->{parent} if $self->_has_parent;
}
Weak references are often a lot more complicated than they might seem. It can be attractive to try and solve a problem by "just weakening" one half of a relationship. Very often, it doesn't solve a problem, it just replaces it with a much weirder or more subtle one.
Here's the meaningless top-level lines-of-code changes performed during my grant work:
151 files changed, 4568 insertions(+), 1371 deletions(-)
Some of that work came from people other than me, but almost all of it is by me and directly the result of the Perl Foundation grant. As each milestone was completed, more and more people seemed to begin to show interest. Dozens of new plugins were written and the whole system was shown to be capable of working in ways I hadn't imagined.
Just before I started work on the grant, there were 284 distributions on the CPAN identifiably using Dist::Zilla, from 45 distinct authors. Now there are 513 distributions from 77 authors, and usage still seems to be growing. I hope the Perl Foundation's investment appears successful to TPF and its donators as it does to me. They made it possible to do a lot of work that would probably have otherwise sat, for a long time, in the "too boring to contemplate doing" pile.
Here's a summary of deliverables included in the original grant proposal:
proper logging facility
reusable testing tools
extensive testing of the core
The logging and testing changes happened first, and what a relief that was. Having regression tests for your old code makes new code easier to add with confidence. It was easy to get Dist::Zilla pretty far without tests, but so much work needed to be done refactoring guts, rather than just adding more features, that tests were crucial to have. As for extensive testing of core, I think coverage is pretty good. There's plenty of room for improvement, but now those additional tests should be easy to write. More importantly, other plugin authors can now use the core testing libraries to test their plugins, which was, previously, an incredibly hairy process. Dist::Zilla was written in part to replace other, very hard to test systems, and I didn't want it to be another one.
Also, with other plugins made testable, I could feel more at liberty to break them by making incompatible changes to Dist::Zilla. It would make tests fail, quickly identifying breakage and making it possible to quickly release a fixed version. I broke backward compatibility a few times, once quite significantly, and I plan to do so again in the future, although in increasingly limited scopes. Having the test system should be a huge boon.
simplification of the command line tool's code
core set of well-known FileFinder plugins
event structure for distribution creation
The command line tools have definitely been simplified. Very few of them do anything more complicated than call a method after some slight argument munging. We haven't gotten many gains from this yet, but I feel it's been a good change. It makes it easier to share behavior like testing, and when we end up with more interesting forms of Chrome (interface delegates), we'll be able to instrument Dist::Zilla without wrapping a command line program.
The core file finders has been a big win, too, by making it easy to refer to a conceptual notion of files. For example, you can write a plugin that will do something to "all the libraries we're going to install" or to "all the executable programs we ship." It also got us the notion of plugin names that get default-configuration plugin objects dropped in when no plugin is given. This has paid off in other areas already, and will probably keep doing so.
One payoff was in the newly-released dzil new command, which can be used to
build a new distribution using Dist::Zilla. This takes the place of tools like
Module::Starter, making it easy to do build new distributions that have none of
the boilerplate that Module::Starter centers around. Because minting new
distributions centers around the same sort of plugin/event system that powers
distribution building, it's easy to add plugins to do all sorts of things, like
import your new distribution into version control. Because Dist::Zilla plugins
can be used across multiple actions, it will be possible to re-use things like
Perl module making plugins later to add new modules to existing distributions.
At first, I was baffled when people wanted a distribution starter tool built
into Dist::Zilla, because Dist::Zilla had been created to avoid needing
boilerplate. Now that I have dzil new working, I can definitely see the
benefits on the horizon.
improved prerequisite handling
improvements for authoring distributions containing XS
Prereq handling was a big win, and it hasn't yet been entirely realized. I wrote Version::Requirements to handle comparing versions to the weird conditions that META.yml allows and used that to replace the lousy "hash of versions" that was being used. Jerome Quelin's excellent AutoPrereqs plugin was updated to interact well with it, and then the libraries that implement META.json 2.0 were built on it, too. It's now extremely easy to deal with the big blob of different kinds of prereqs, and once a few changes are made to CPAN::Meta, it will become even easier.
The XS improvements made by me have been minimal, and boil down to two things: I gave Ævar some tiny amount of help while he worked in the Makemaker::Awesome plugin, and I made a large set of notes and todo items for making XS distributions easier to manage in the future. It will take quite a while to get through those notes, but the whole system will benefit. In the meantime, a number of XS dists have been released using Dist::Zilla, showing that it's now possible to do.
documentation: improved new user's guide
Well, there is one! dzil.org now exists, and has a tutorial and some other helpful information. The tutorial is designed to be easy to expand, and I'm sure it will over the next few weeks and months. I also hope to find a way to distribute it, in somewhat altered form, as installable Pod in a CPAN distribution. I'll also be giving presentations on Dist::Zilla at YAPC and OSCON, and once those are complete I'll update the slides with feedback received from the audiences and I'll publish the slides -- hopefully with an audio track.
So, all told, I think this was a very successful bunch of work. It's already very clear, too, what the next big obstacles to overcome are, and I look forward to working on them to make Dist::Zilla more powerful, more flexible, and easier to maintain.
I'm very pleased to report that the last to-do item from my 2010Q1 Perl Foundation grant to improve Dist::Zilla is complete. That item was an improved dzil new command, described like this:
event structure for distribution creation
In other words, plugins will be able to attach more behavior to distribution creation, to create new source code repositories, start files, and so on.
If you already use Dist::Zilla, you know how this should work: more roles for plugins to consume! The "make a new dist" method looks something like this:
$_->before_mint for $self->plugins_with(-BeforeMint)->flatten;
$_->gather_files for $self->plugins_with(-FileGatherer)->flatten;
for my $module (@modules) {
$module->{minter_name} ||= ':DefaultModuleMaker';
my $minter = $self->plugin_named($module->{minter_name});
$minter->make_module({ name => $module->{name} })
}
$_->prune_files for $self->plugins_with(-FilePruner)->flatten;
$_->munge_files for $self->plugins_with(-FileMunger)->flatten;
# ... code to write files to disk, etc ...
$_->after_mint({ mint_root => $dir })
for $self->plugins_with(-AfterMint)->flatten;
You can configure as many different sets of plugins as you like and drop them in ~/.dzil/profiles so that you can create dists with different build types. Later, we'll make it possible to add new content to an existing dist. For now, we have enough new behavior to reasonable use dzil new for new work, and we have enough plugin roles to implement VCS integration. Here's what it looks like in action:
All of this is available in Dist-Zilla 2.101230, which I've just uploaded to the CPAN.
I think I've finished all the pages I meant to write for the Dist::Zilla Choose Your Own Tutorial. It covers a bunch of Dist::Zilla topics like creating a new dist, converting an existing one, and adding new behavior to your configuration. The "choose your own" format should make it easy to add new content, like a wiki, but I like the almost-linear feel of the format.
I also really like being able to say that The Perl Foundation paid me to write a Pod-based Choose Your Own Adventure system.
With this done, the only thing left to do is finish overhauling the way that
dzil new works! I predicted this work could be done within a quarter (of a
year) and I think I'm definitely on track for that.
This weekend, David Golden and I holed up in the Pobox offices and worked on the final draft for v2 of the CPAN Meta specification. It's the document that describes what goes in META.yml -- or, now, META.json. This was not very exciting to many people, but we were both really excited to get it done, and hopefully the final v2 spec and support libraries will be out in a few days.
There's a big benefit for Dist::Zilla here. One of the things we produced is a class that handles organized sets of prerequisites. For example, it can track the modules needed to configure before installation, the modules recommended to improve your testing, and the modules required to use the code. These sets can be merged, inspected, and marked as "closed for new changes."
Dist::Zilla::Prereq already does some of this, but not very well, apart from the central bit built atop Version::Requirements. The new library is very similar in purpose, but much more useful. It will replace the guts of Dist::Zilla::Prereq, greatly improving the usability of the prereq system.
In other good news for Dist::Zilla, I've made some good conceptual progress on
a few things after some talk with David. For one, dzil new should be getting
some real improvements soon. David wrote yet another "new distribution
starter" tool some time ago, and I thought it had a number of really nice
features. I talked over my problems with writing a new engine for dzil new
and we ended up with a plan that I think I like.
We also talked about how to improve MakeMaker and ModuleBuild by allowing arbitrary hunks of code to be added to the output. This is something I've wanted to do for a while, and I think it will really improve those plugins without requiring users to write custom templates. We talked about a few strategies for doing this, and I'm going to ask for some more input from the mailing list tomorrow or so.
For now, though, I'm really tired. Tomorrow, I'm back at the office, but this time to start churning out some more code for work. I need to be bright-eyed, bushy-tailed, and sharp-witted, so I think that right now I'll go crash.
I spend a lot of time dealing with character encoding issues. Getting them just right is important and sometimes tricky, but getting them wrong makes you look like an amateur. This paragraph from a blog post about PHP and encoding made me smile.
Dealing with character sets and encodings is tough. As long as you're dealing only with English texts you in a luxury situation and can mix utf-8 and iso-8859-1 encoded texts and most (all?) of your tests will work. Some of your users, like me, with strange names ("Schlüter") will be annoyed as your application breaks them ("Schlüter"), but these will be edge cases.
This morning, I uploaded Dist-Zilla-2.100860-TRIAL.tar.gz. It's the first
candidate for what will be Dist::Zilla v2. Notice that TRIAL in the file
name? Of course you do. That tells PAUSE, the CPAN upload system, that this
release is just a test release, and shouldn't be indexed as the latest release.
It prevents users from automatically upgrading to this version, even though
it's newer than the last one. Usually, when you see someone trying to send
this message to PAUSE they do it by setting a version like 1.234_001. The
underscore tells PAUSE that it's a trial release.
This is, I think, a pretty lousy way for it to work. Because you want your
version to be a number, but because (for uninteresting reasons) you want the
first assignment to $VERSION to keep the underscore, this magic incantation
shows up in your code:
our $VERSION = '1.234_001';
$VERSION = eval $VERSION;
Why? Well, a lot of people use it without knowing, but it's to get $VERSION
holding a number with the value 1.234001, because although 1.234_001 would be
that as a literal in Perl, the string form has the numeric value 1.234. People
also get confused about the numbering. The version after 1.234_001 is not
1.234, and generally it shouldn't be 1.235, either. It should be either
1.234002 or 1.234_002 or 1.235000.
Finally, because of that eval, you can't even look at $VERSION to
determine whether it was a trial release. The underscore will be gone.
If you think this is really stupid, then you and I agree. Now, PAUSE only
looks at the tarball name. You don't actually need to change your package
versions -- it's just an easy way to get that underscore in there. If your
tarball's version appears to differ from your package versions, it can be
confusing. You will get exactly the same behavior from PAUSE if your
distribution's tarball's filename has TRIAL in it, and all the version number
related nonsense goes away.
Dist::Zilla now makes it very easy to use this system:
$ dzil release --trial
This builds your distribution just like normal, then packs it up in a tarball
with the right kind of name. I'm toying with the idea of adding a $IS_TRIAL
package variable if you use the (hypothetical) [PkgTrial] plugin, but I can't
really see much use for it yet.
Dist::Zilla v2, starting with this trial release, has quite a few significant
changes, quite a few of which are not backward compatible. I've tried to make
sure that most users won't notice the change, but if you're doing much other
than using @Classic, you probably will.
This makes the name match up more clearly with what it does, because although it defaults to gathering from the dist root, it can gather an arbitrary directory. Also, with FileFinder plugins showing up all over the place, the temptation to try to find "AllFiles" just seemed too great.
This plugin was a stupid idea. It was one place to set up directories to install as sharedirs (q.v. File::ShareDir) and executable programs that go into your path. These are really different, but I had a clever (read: terrible) mechanism to make them seem similar. I ripped out the terrible mechanism a while ago, and that just made things more complicated. Now they're different things, and they make a lot more sense.
Pod::Coverage, even with CountParents and Moose and TrustPod, doesn't quite cut it for me. There's no reason that everybody who wants one will want the other. I'd like to use the coverage tests again someday, but right now I can't -- and anyway, I know plenty of people who say they'll never want them.
Plugins performing FixedPrereqs would return a hashref of prereqs to register. This was fine when there was only one kind of prereq. Now that we handle different kinds (like prereqs for build-time only) I needed more flexibility. Also, Pedro Melo wanted to write a plugin that would up all prereqs to the latest version from CPAN. Someone else wanted to require the version installed on his computer, since that what he tested with. FixedPrereqs couldn't do this, either, and I didn't want to add the proposed PrereqMunger phase and role.
Instead, plugins that perform the PrereqSource role will be given a chance to register their prereqs just before prerequisite finalization. They can register different kinds of prereqs, and they'll be able to inspect, clear, and tweak existing ones.
You can look at the Changes file for version 2 to see what else is new.
Well, there are the unfinished code todos for the TPF grant:
(Those are all the remaining items to do for the code half of the grant. Isn't that exciting?)
There are quite a few more todo items in the repo's ./todo directory and in
the RT queue for the distribution. Most of these will have to wait until I've
completed the grant work. (In fact, most of the best improvements for XS
authoring will as well, I think. The useful ones are going to require much
more extensive changes than were originally imagined.)
With the bulleted items above done, I'll be able to move on toward the new documentation and learning material. Once that is done, I've got my big pile of ideas to work through. My goal is to work through them steadily, keeping backwards compatibility when possible. Although I'm generally extremely conservative about breaking backcompat, I'm not going to worry about it too much with Dist::Zilla, yet. It's too young, too complex, and too fast-moving for me to get everything right the first (or second, or third...) time just yet. Instead, I'll try to work on big new features in branches and when releasing things that break backcompat, I'll bump the major number.
I might produce some sort of documentation, too, that indicates how likely to change any given part of the system is. I've got to think about that more.
Finally, because there was a bit of confusion about this on IRC earlier: Dist::Zilla's master branch in git is not stable. It is where the main development occurs, and new, slightly-unstable features may land at any time. For a stable Dist::Zilla, start with a tagged release. Eventually, there may be a maint branch, but I don't see that happening any time really soon.