There are various ways to trigger a Jenkins job from an SCM like Bitbucket, but what I want to do specifically is trigger a build using the branch that is the source of the Pull Request.
Up to now, we have used the Bitbucket Pull Request Builder, but it is very flaky and unreliable, and not supported well.
https://wiki.jenkins-ci.org/display/JENKINS/Bitbucket+pullrequest+builder+plugin
Bitbucket do supply quite good features in terms of Webhooks, which when used with the Jenkins Git Plugin, do allow for triggering of builds based on various Bitbucket events (eg a Pull Request update).
There is also the Bitbucket Webhook plugin, but again that doesn't offer much in terms of dynamically choosing the branch you want to build.
https://wiki.jenkins-ci.org/display/JENKINS/BitBucket+Plugin
However, what this seems to do is trigger a poll of the repo, where is then tries to build any branch that is different from the main branch.
Our use case is that we allow developers create their own branches, for which they then create Pull Requests to the development branch.
There doesn't seem to be any way to trigger a build that uses the developer created branch as the build branch (other than the aforementioned Bitbucket Pull Request Builder).
Am I right or wrong in this?
We create a jenkins job for each developer, and then use a parameterised Jenkins build, you just put in the branch name and click build. I know it's not completely automated, but it works well enough. Then you can use the Stash notifier plugin to signal back to Bit Bucket that the build is good - you get a nice green tick in Bit Bucket.
I've implemented such a solution at a customer's company and I think the BitBucket pull request plugin is great and answers most requirements.
You are correct, the BitBucket pull request plugin is your best shot, otherwise it may require you to develop yourself a plugin but I really don't think you should as the plugin allows you everything you can think of.
In your case, you can configure the plugin to build only specific branches or all branches.
Let's say a developer checks out a new branch from the development branch, he then develops and pushes a commit, the Jenkins job which is configured as a multi repository multi branch pipeline scans the BitBucket team every 10 minutes for example, and when it identifies that a new code has been pushed, it automatically creates a new job for the pull request and a new job for the branch.
When the pull request job finishes, it notifies BitBucket with the status of the build (success, failure) and then the reviewer sees a green mark in Confluence, meaning that the job has finished properly with the new code, he then reviews the code - this gives him two factors: the first is that the code is playing nice with the code at the development branch and the second factor is his own code review, if he approves the code then the new code is merged to development, the pull request is being automatically closed (or not - it's configurable in the plugin) and next time the scan runs, it identifies the push to development and triggers a new run.
If you look on the plugin configuration you can configure which repositories within the team and which branches within each repository should be automatically built.
You can check the pipeline I wrote for that customer in my Github account.
I hope my answer helped you.
We have also encountered this issue, and eventually found that the Generic Webhook Trigger fulfills all our needs perfectly.
This plugins enables you to define a trigger based on a bitbucket webhook without the need to define an SCM step in your job, it also enables to easily extract any needed parameters from the Event Payload sent by Bitbucket.
It has excellent support for Freestyle Jobs, Pipeline Jobs and Job DSL.
Only tricky part is to match webhooks to Jenkins Jobs, which is not done automatically using the repository URL like other triggers, but we used the built in token mechanism to control which jobs are triggered by different repository webhooks which gave us the needed mapping.
Overall we have been using this plugin for several months and it works perfect.