Interesting LinuxConf2018 talks

I have been fortunate to watch several talks:

1 Large Memory Management issues: Performance, Fragmentation, Movable objects and Huge Page overhead

Talked about a new type of RAM coupled with FPGAs. For more information please see: https://www.rambus.com/blogs/smart-data-acceleration-with-fpgas-and-dram-2/ or watch the talk mentioned above.

2 Another good talk is 

Mastering Chaos - A Netflix Guide to Microservice With supplemental materials at

https://www.infoq.com/presentations/netflix-chaos-microservices here he mentions the analogy of the human body and how many systems work together but they all interface with eachother. Netflix is la large company who migrated to the Amazon cloud. They look at actors frameworks seriously.

3 Devoxx gRPC vs REST: let the battle begin!

This talks about monolythic architecture and compares it with new architectures such as those who use Microservices. The authors work with arcade games and describe their environment.

4. Distributed Architectures: Microservices and Messages is another talk I recommend. Keep in mind there could be processes that are external and internal communicatioms.

5) caching at Netflix: The hidden microservice by Scott Mansfield by the strange loop.

6. Kernel Recipies 2017 An Introduction to the DRM subsystem

7. Yocto Linux Primer

8. VHDL basics by Intel FPGA.

Upcoming paper

I have submitted a project proposal to IGI and got my paper proposal approved.

Dear Prof. Wahlstedt, Thank you for submitting your chapter proposal, "Towards Optimal GPU Performance for Real World Savings," for the upcoming book, "Harnessing the Internet of Everything (IoE) for Accelerated Innovation Opportunities." After reviewing your recent proposal, I believe the topic of your proposed chapter can be consistent with the theme of this book and could make a significant contribution to this project. I suggest you to review the focus of the paper. It seems that concurrency and parallel programming in FPGA can only marginally be on the scope of the IoE and the book. However, harnessing the power of using software to save energy that is consumed by embedded IoT devices can be a hot topic. Moreover, a different title for the chapter can help on focusing the paper on a subject with more relevance to the book. For instance, I may suggest a possible title such as: “The importance of software on energy savings in IoT devices”; or “Software development techniques to improve energy savings in IoT devices”. I therefore encourage you to begin preparing your full chapter and submit a copy before Monday, April 30, 2018 at the following URL: https://www.igi-global.com/submission/submit-chapter/?projectid=26e0077b-e04f-40b7-aa3c-e5149f2fa97e Please note that you will be asked to create an account prior to uploading your chapter to the system. This is to ensure the security of your work and to assist you in organizing your materials for submission, receiving and providing peer reviews, and making any necessary revisions to your chapter. For information on creating and accessing your Web account, please see our tutorial at www.igi-global.com/publish/contributor-resources/book-submission-system/video-guide/#creating-a-user-account. The following documents are provided by the publisher, IGI Global (www.igi-global.com), to aid in the writing of your chapter: Ensure your chapter has been professionally copy edited It is crucial that professional copy editing is conducted prior to submission to ensure proper use of the English language, proper grammatical structure, and correct spelling and punctuation. Submitted chapters are considered to be in their final form and ready for publication as is. We recommend eContent Pro for copy editing (currently offering 25% discount): https://www.econtentpro.com/copyediting. Details to keep in mind for your full chapter submission: www.igi-global.com/publish/contributor-resources/before-you-write/ Image guide: http://www.igi-global.com/publish/resources/image-guide.pdf Your adherence to the guidelines provided in these documents is very important. Should you have any questions regarding your proposed chapter, please do not hesitate to contact me, Jorge Semião, at jsemiao@ualg.pt. I look forward to receiving your chapter! IGI Global eEditorial Discovery®

You have received this email because you are associated with a project in the IGI Global eEditorial Discovery® system. Adjust where notifications are sent by adding or updating your primary email address at https://www.igi-global.com/account/e-mail/ (login required). Please contact cust@igi-global.com for assistance.

Actor frameworks

Actor frameworks are composed of actors which are entities that are based on an application programming interface. An example of an API is the C++ programming language. It has public methods that are exposed to public entities. An entity could be another actor such as a person or device capable of telling other actors information. There is a method in actor frameworks known as ask which not only tells but asks for a reply. Sometimes this information in the reply can be anticipated. The API is to map something which is useful for a business. For example a teapot serves as an API because it serves a purpose. Another feature could be the entity such as the barista or the credit card buyer. The teapot can hang from a hang from its handle. The handle could be an API for the barista. Microservices are a part of the spring framework and are the small processes of working between the entities. I am working on taking classes about spring, microservices and the actor framework. I am particurally interested in how the spring framework could generate microservices. I have been playing with Bonita Studio which is an open source business modeling software. It uses REST and SOAP API. I think it is helpful to learn definitions which could help us with the project
Bean Factory: the container which instantiates a number of beans
Java beans: an object that encapsulates many objects into one java bean object
Spring: a dependency injection framework for java. It is thought of as a replacement to java enterprise edition.
Jhipster: a microservices generator for Spring

VHDL can be thought of as a hardware description language. It can be thought of as similar to HTML in that they are description languages. Jeffrey Snover describes desires state configuration as a “let it be so” programming language that does not require intimate knowledge of the computer systems to suggest changes to make them compliant with the desired state. The video cards have their own bios in them. By using a hardware description language as inspired from those which affect a web browser’s DOM. Not all hardware is alike and therefore it is like a browser’s DOM.

I watched a video that motivated me on the project. As the baby boomers in the United States age, they need more assistance driving. Neuromuscular diseases can restrict them from being a part of society. As people were able to drive automobiles to work instead of walking they had more of a purpose in life. The life expectancy in the time our country was founded was about 44. Now it is not unheard of to be 90 or 100. I believe that finding a purpose in life can be a meaning to get out of bed in the morning. Augmented reality is helpful in breaking complex situations into less complex situations. Factory work can be busy and workers are often wearing gloves. By displaying helpful information in the clearest way possible can be very helpful. With a faster, clearer viewing monitors it can help them in their persuing of happiness. We have come a long ways from the original computers that were heavily pixelated.

VHDL appears on first glance as a more complicated version of the programming language that interfaces with the Arduino board. The original keyboards had a special chip that could control them that would go on the motherboard. Now keyboards use USB where the keyboard scanner is implemented in hardware.

I am excited to work on the memory management for the driver.

Proposal for Xorg Foundation's Nouveau Device Drivers for Vulkan

Abstract

Several web magazines have questioned the feasibility of drivers for Khronos' Project Vulkan. This article is to address concerns presented by Michael Larabel, the founder and principal author of Phoronix on his article Don't Expect An Open-Source NVIDIA Vulkan Driver Anytime Soon. This has presented me with an inspirational questioning where I felt as though I could design a driver with the best intentions of the Khronos Group and others who have inspired me. I hope this article provides some hope that the entities of the open source community have come together to work with one another to make a project that withstands the test of time in it's functionality and inspiration to generations to come.

Please see preprint at: https://www.researchgate.net/publication/322486630_Proposal_for_Xorg_Foundation%27s_Nouveau_Device_Drivers_for_Vulkan

Architecture update

One question I asked myself is what level should this driver run on the Linux operating system. The GPU historically runs embedded on the Linux Kernel as a device driver. However there are several reasons why I am debating putting this into the user space. The first reason is that according to Marshall McKusick is that it is faster to use a subprocess than a lot of hardware interrupts. Also it is possible to cause a kernel panic.

I looked on Docker and it appears to run on built in containers rather than on LXC. I also would give runc a greater look.

I looked into SR IOP and found that AMD produces a version similar to what I wish to do called GIM. However it is specific to AMD. I wish to make it generic. However a reddit thread pointed out some issues. It is only available in enterprise GPUs because it uses ASIC. I hope to use FPGA and asking researchers on researchgate if this is theoretically possible. I found these two articles that I am reviewing:

A Albanese, PS Crosta, C Meani, P Paglierani - ICN 2017, 2017 - researchgate.net

https://www.repository.cam.ac.uk/bitstream/handle/1810/260326/Zazo_et_al-2016-IEEE_ReConFig-AM.pdf?sequence=1

Works Cited

“Docker Frequently Asked Questions (FAQ).” Docker Documentation, 7 Jan. 2018, docs.docker.com/engine/faq/.

“How Is Docker Different from a Normal Virtual Machine?” Stack Overflow, stackoverflow.com/questions/16047306/how-is-docker-different-from-a-normal-virtual-machine.

McKusick, Marshall Kirk. “Introduction to the FreeBSD Open-Source Operating System.” O'Reilly | Safari, Addison-Wesley Professional, June 2015, www.safaribooksonline.com/library/view/introduction-to-the/9780134306049/FBSD_02_02.html.

“No SR-IOV Support on RX Vega. Confirmed by AMD • r/Amd.” Reddit, www.reddit.com/r/Amd/comments/6tpkf8/no_sriov_support_on_rx_vega_confirmed_by_amd/.

Opencontainers. “Opencontainers/Runc.” GitHub, 15 Dec. 2017, github.com/opencontainers/runc.

I found some good material on the construction of GPUs and am reviewing them. These include:

Farber, Rob. Parallel Programming with OpenACC. Morgan Kaufmann, 2017.

Garney, Ben, and Eric Preisz. Video Game Optimization. Course Technology Cengage Learning, 2011.

Kim, Chul-Woo, et al. High-Bandwidth Memory Interface. Springer, 2014.


Sarbazi-Azad, Hamid. Advances in GPU Research and Practice. Elsevier, 2017.

Shackleford, James, et al. High-Performance Deformable Image Registration Algorithms for Manycore Processors. Elsevier/MK, 2013.

• Synopsis. A short summary. My contribution would be working on the Vulkan driver which would be measurable with video streaming benchmarks. This would demonstrate appropriate buffering, bit rate, and lag time. I would expect it can handle YouTube but it might be a little too soon for other video streaming such as Skype for linux. The benefits are as fewer Linux distributions support Adobe Flash perhaps to licensing concerns, there is html5 which utilizes javascript. Javascript uses concurrency or parallelism which could have better performance on a system which uses low level FPGA which the Vulkan API is designed to do. Vulkan is designed to be cross platform meaning that it should work with AMD as well as Intel and Nvidia video cards whether they are embedded or are discrete. Although there is primitive support for Arch Linux, the GPU profiling tool is complex for Nvidia.

• Benefits to the Community. What novel technologies or approaches will be demonstrated?  Windows is investing time into Direct X 12. This was shown to be a challenger to the Vulkan API. By investing in Vulkan we can gain a strong backing from the gaming community as well as business professionals who use embedded GPU in their laptops. As of right now, steam works on Linux but it also works on Windows.

•  Deliverables. Give a brief, clear work breakdown structure with milestones and deadlines. Make sure to label deliverables as optional or required. You may want plan to start by producing some kind of whitepaper, or planning the project in traditional software engineering style. Work should include 

  • • investigation I have taken several courses on Coursera on FPGA as well as a member of several computer organizations which provide Safari, Lynda, books24/7. I have done a great amount of research on this topic while as an undergraduate and wish to make Linux better. I have resources as a student should I have difficulty through the ACM/IEEE/ISACA and academic as well as community libraries. 
    • programming I would make every effort to have a working prototype by mid September.
    • documentation: I would post on my blog XorgRobWahl and be available for questions as an active community of the Linux/xorg Community.
    • dissemination I am open to ideas from the community about how to proceed in this regard
  • Description. A list of project details (rough architecture, etc).

  I have participated in putting patches on https://github.com/KhronosGroup/Vulkan-LoaderAndValidationLayers

  
  

  
  

  I believe that a wrapper service is more appropriate than trying to restart the project. Right now there are many implementations and I would try to generalize them to use the Vulkan API including SPIRV. I like their advanced blend modes, 2d shapes, and multi-GPU approach. I also like their effort to go with an open standard.

  
  
  I also like SR-IOV and how it appears to be a clean way of communicating with PCIe components. Although not all GPUs are PCIe based, it provides some interesting ideas.

• 
  

  
  

  I propose following design changes. I was concerned about using a python runtime. I have supported an older blackberry server which used Java and was subject to break should the version of Java be upgraded. I would try to make the most future endurance version I could. would make a system initiated version of a daemon which uses C as a primary language. C and C++ are two of my strongest languages both having access to memory access on the computer. From C or C++ it is possible to initialize threads. I would then use the existing Khronos link as enclosed which provides an API for manipulating the FPGA which are used with the video cards. AMD was a leader in project Vulkan and so I would use this to generalize a CUDA version to work with Nvidia. I do not see Vulkan as a replacement to openGL. I have watched a Usenix security videos on shared memory execution. This involves multicore execution. There are often multiple processor cores in the computer. One might be processing the driver software and the other a piece of malicious code. The cores often have a shared L1 cache. This is defined in The common CVEs

  • Variant 1 (CVE-2017-5753): Mitigations are provided with the security update included in this bulletin. NVIDIA expects to work together with its ecosystem partners on future updates to further strengthen mitigations.
  • Variant 2 (CVE-2017-5715): Mitigations are provided with the security update included in this bulletin. NVIDIA expects to work together with its ecosystem partners on future updates to further strengthen mitigations.
  • Variant 3 (CVE-2017-5754): At this time, NVIDIA has no reason to believe that Shield TV/tablet is vulnerable to this variant.
  Some might argue that security is a lower priority than timely delivery. I believe from working in IT the way to make work for it to be perceived as a viable product it should have every effort to be failure resistant. Simplicity is important and Nvidia presents the idea of approaching zero overhead. This idea is expansive but I plan on making the Vulkan footprint as small as possible for security and maintain ability. A possible suggestion is to use the Rust programming language developed by Mozilla. Rust is stylistically like C++ but has memory and thread safety built in. 

  • Related Work. A list of other people's work. Could be as simple as a URL with one sentence description. Be sure to explain how the proposed work is different from similar related work.

  Works Cited

  FuturemarkTV. “Compare Vulkan and DirectX 12 with 3DMark.” YouTube, YouTube, 23 Mar. 2017, m.youtube.com/watch?v=E3t7JwdF6W8. 

  gameranxTV. “What Is Vulkan & Why Should Gamers CARE?” YouTube, YouTube, 30 Apr. 2016, m.youtube.com/watch?feature=youtu.be&v=dvioALNs_Bc. 

  Mark Kilgard, Graphics Software Engineer at NVIDIA Follow. “NVIDIA OpenGL and Vulkan Support for 2017.” LinkedIn SlideShare, 2 Aug. 2017, www.slideshare.net/mobile/Mark_Kilgard/nvidia-opengl-and-vulkan-support-for-2017. 

  Tedhudek. “Overview of Single Root I/O Virtualization (SR-IOV).” Microsoft Docs, docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-single-root-i-o-virtualization--sr-iov-.

  USENIXAssociation. “USENIX Security '17 - Strong and Efficient Cache Side-Channel Protection...” YouTube, YouTube, 15 Sept. 2017, m.youtube.com/watch?v=4_-c7ueHHqc. 

  “WTF Is Going on with DX12 and Vulkan?” YouTube, 19 Sept. 2016, youtu.be/r0fgEVEgK_k.

  
  
  Similar but incomplete versions consist of 
  

  
  

  https://github.com/NVIDIA/nvidia-docker
  
  
  This is a project by Nvidia using the open source project Docker. However this is designed specifically for FPGA on Nvidia video cards leaving out AMD and Intel. This is not generic for other computer architectures. It also is not designed as a daemon because it Docker runs on the operating system with native execution.
  
  
  Also similar is Vulkano which could be found at  https://github.com/vulkano-rs/vulkano . The authors claim that their software is not ready for production machines. They try to rewrite Project Vulkan and do something similar but better including the pieces that interact with the FPGAs on the GPU. This is a huge task.