This week
This week my pull request for the logic refactor of recording api was finally accepted after a number of code review changes.
Most of my work this week involved researching what I should do next, and how to implement it. I began doing some research on authentication schemes, and have a clear vision for how authentication should be carried out between client and server. After some conversations with Michael Tom-Wing, he noted that it would be better to think about how the server could be broadcasted onto a network and then worry about an authentication scheme later, which will probably rely on shared keys instead of a user/password model. I began looking into what would be the best library for broadcasting, and I'm still researching. The library that seems the most promising thus far is python-zerconf, which is compatible with both avahi and bonjour, so it should be usable by Windows and OSX as well. I was looking at pybonjour, which is also cross-platform, but does not have support for python 3.
I also thought about a more consistent way to throw and present errors in the REST framework. My plan is to have a table of generic messages associated with each http error in the http error class (400, 404, ... etc.). When an endpoint wants to throw an http error, it then supplies any endpoint specific message it wants to send to the constructor.
I was also sick with the flu Saturday night into Sunday, so I was unable to attend the meeting.
Next week and beyond
I have a clear direction for the last few weeks working on getting our Freeseer REST server to broadcast on a network, so that will be my main focus, as well as small changes to the way errors are thrown by the REST server. As well, I plan on writing a small document about what a developer will need to take note of when developing new endpoints. If I run out of things to do, I always need to expand test coverage. I've modified my proposal to take notes of these changes.
Until next time,
Brenden Doyle
Monday 27 October 2014
Sunday 19 October 2014
UCOSP Update, Week 6
This week
This week I had a pretty crazy week with midterms. I managed to make changes based on Michael's code review, but I haven't had too much time to think about the code.
Essentially the biggest change this week I made was in the test framework. Our tests require a number of temporary folders to stick files necessary to carry out tests. Before, we would create these temporary folders and delete them on our own, but now the test framework makes use of a pytest fixture called 'tmpdir' that takes care of creating any temporary folders we need to carry out tests and deleting them automatically.
Other than that, there were minor changes here and there to the test framework.
Next week
The main thing I am thinking of right now is changes that can be made to the way we handle errors in the framework, and have a standardized way to handle errors that all new endpoints can adhere to, and how to handle authentication. As soon as my current pull request is closed, I can start to work on either of those.
Until next week.
This week I had a pretty crazy week with midterms. I managed to make changes based on Michael's code review, but I haven't had too much time to think about the code.
Essentially the biggest change this week I made was in the test framework. Our tests require a number of temporary folders to stick files necessary to carry out tests. Before, we would create these temporary folders and delete them on our own, but now the test framework makes use of a pytest fixture called 'tmpdir' that takes care of creating any temporary folders we need to carry out tests and deleting them automatically.
Other than that, there were minor changes here and there to the test framework.
Next week
The main thing I am thinking of right now is changes that can be made to the way we handle errors in the framework, and have a standardized way to handle errors that all new endpoints can adhere to, and how to handle authentication. As soon as my current pull request is closed, I can start to work on either of those.
Until next week.
Tuesday 14 October 2014
UCOSP Update, Week 5
Hello everyone.
What I worked on this week.
Implemented shelve in recording.py
This week I began working on an issue where I was trying to figure out the best way to approach tearing down APIs when the rest server stops working. For example, for the recording api, when the server shuts down, the recording api fires some code to write any references to videos to disk. There were a few issues to this:
1. The way to get this code to fire was very hack-ish. It relied on firing code when a kill signal was sent. Furthermore, this code was called from the recording api, and not from some more central location like the server.py class. If we had multiple apis that required some sort of teardown when the server shutdown, it would be better to have a more top down approach. Preferably something that flask provided.
2. There was really no default option provided by Flask to fire teardown code when the rest server shutdown. Since flask provided a number of teardown decorators (post-request teardowns etc.), but provided no teardown decorator that accomplished what I was trying to do, I figured that flask probably actively discouraged this style of teardown.
3. No references to recorded videos were written to disk until after the server shutdown. I was worried that if anything were to go wrong during the shutdown that this teardown code didn't fire, all information in our media dictionary would fail to persist. This was worrisome.
So I decided to make changes to the rest server and recording api so that no teardown code needed to be fired on shutdown of the REST server.
Instead of writing information to disk on shutdown of the REST server, I decided to use a persistent object library called "shelve". Basically you open a "shelve" object, and can treat it as a dictionary (with a few limitations) and when you want your changes to persist, you call sync() on the shelve object.
So, that works well, and no server shutdown teardown code is required for recording api to work properly anymore.
Implemented py.test fixtures in test_server.py
I had to make a number of changes to our REST server's tests so they would work with our changes, and I realized that since I was thinking about the testing framework, that it would be a good time to change our xunit style setup and teardown methods to py.test fixtures. So now test_server.py uses only py.test fixtures to create a baseline for our tests to run against.
Next week
Two things I'm thinking of all the time is the REST server's error framework, and authentication. So depending on when these changes I made this week get merged in, I will work on either of these. I will probably tackle authentication because I already have a lot of work stashed for implementing this. But I am always looking at ways that I could improve the error framework and will be noting things incrementally.
Until next week!
What I worked on this week.
Implemented shelve in recording.py
This week I began working on an issue where I was trying to figure out the best way to approach tearing down APIs when the rest server stops working. For example, for the recording api, when the server shuts down, the recording api fires some code to write any references to videos to disk. There were a few issues to this:
1. The way to get this code to fire was very hack-ish. It relied on firing code when a kill signal was sent. Furthermore, this code was called from the recording api, and not from some more central location like the server.py class. If we had multiple apis that required some sort of teardown when the server shutdown, it would be better to have a more top down approach. Preferably something that flask provided.
2. There was really no default option provided by Flask to fire teardown code when the rest server shutdown. Since flask provided a number of teardown decorators (post-request teardowns etc.), but provided no teardown decorator that accomplished what I was trying to do, I figured that flask probably actively discouraged this style of teardown.
3. No references to recorded videos were written to disk until after the server shutdown. I was worried that if anything were to go wrong during the shutdown that this teardown code didn't fire, all information in our media dictionary would fail to persist. This was worrisome.
So I decided to make changes to the rest server and recording api so that no teardown code needed to be fired on shutdown of the REST server.
Instead of writing information to disk on shutdown of the REST server, I decided to use a persistent object library called "shelve". Basically you open a "shelve" object, and can treat it as a dictionary (with a few limitations) and when you want your changes to persist, you call sync() on the shelve object.
So, that works well, and no server shutdown teardown code is required for recording api to work properly anymore.
Implemented py.test fixtures in test_server.py
I had to make a number of changes to our REST server's tests so they would work with our changes, and I realized that since I was thinking about the testing framework, that it would be a good time to change our xunit style setup and teardown methods to py.test fixtures. So now test_server.py uses only py.test fixtures to create a baseline for our tests to run against.
Next week
Two things I'm thinking of all the time is the REST server's error framework, and authentication. So depending on when these changes I made this week get merged in, I will work on either of these. I will probably tackle authentication because I already have a lot of work stashed for implementing this. But I am always looking at ways that I could improve the error framework and will be noting things incrementally.
Until next week!
Saturday 4 October 2014
UCOSP Update, Week 4
Hello readers. Another busy week for me with many updates.
Changing priorities
Last week I posted my project proposal, and I've already received some feedback which made me decide to change one of my priorities for the UCOSP project. Worried that if I focused solely on the API framework I would run out of things to do this semester, I considered implementing server-client video streaming, but I was told that there was even more work that could be done to the api framework than what was stated in my proposal. Instead of implementing video streaming, I more than likely will be implementing a way to broadcast our Freeseer REST server over networks, and anything else I discover should be done.
Until next time.
-Brenden
What I worked on this week.
My first pull request (link to PR) was merged this week. I learned a lot about how to format your PR commits and summaries properly so it will be accepted. So the next PR should go a lot smoother than this one (apologies to mtomwing :) ).
This week I worked on refactoring the recording api endpoint functions and fixing logic (link to PR). The reason that I didn't do this in the initial framework refactoring PR is that I essentially treated these methods as black boxes and only changed what needed to be changed to decouple the recording api routes from the server.py. Now that that work was complete I could focus on the internal logic of the endpoint functions. I made a number of changes, cleaned up if/else nests, and fixed some faulty logic. These changes are ongoing but should be merged very soon.
Finally, this week I began researching and implementing authentication for the Freeseer server. I read up on REST best practices, and tutorials on how to implement authentication in Flask, and I now have code implementing authentication in the REST server stashed and ready to go into its own branch once my most recent open pull request is merged. In its current state, if one tries to send requests via curl to the API it will give you an authentication error unless you provide -u admin:admin in the command. I need to think more about how we should go about authenticating many users, but it will probably involve tokens generated by the Freeseer server. I'll be focusing on this all week.
Last week I posted my project proposal, and I've already received some feedback which made me decide to change one of my priorities for the UCOSP project. Worried that if I focused solely on the API framework I would run out of things to do this semester, I considered implementing server-client video streaming, but I was told that there was even more work that could be done to the api framework than what was stated in my proposal. Instead of implementing video streaming, I more than likely will be implementing a way to broadcast our Freeseer REST server over networks, and anything else I discover should be done.
Next week
Next week I will finish implementing authorization for the REST server, and finish the logic refactoring and fixes to the recording api. Looking forward to another busy week.
Next week I will finish implementing authorization for the REST server, and finish the logic refactoring and fixes to the recording api. Looking forward to another busy week.
Until next time.
-Brenden
Subscribe to:
Posts (Atom)