emote Desktop Services in Windows Server 2012 is awesome. With highlights like huge performance improvements and an incredibly simplified deployment process, you’re going to want to see what this can do for your business and you can, for free! Microsoft has the Windows Server 2012 Release Candidate available which you can download and install today. I’ll show you how you can set up several scenarios.

1. Quick and Easy, RemoteApp on a single server
2. Quick and Easy, RemoteApp using three servers
3. Adding a Gateway and Configuring Certificates
4. Adding a Licensing Server
5. Adding a Windows Server 2008 R2 RemoteApp source

Some articles I intend to be adding soon (more of a note to myself really)…

1. Configure a Virtual Desktop Infrastructure Pool and PVD
2. Building and Maintaining VDI Personal Virtual Desktops
3. Delivering RemoteApp to end users via RSS Subscription

Let me know if you want to see something added to the list! Throughout these guides there are a couple acronyms I’ll be using pretty regularly and my servers will tend use them in their names because I like to name my servers after the roles they will be delivering. There are three fundamental roles to an RDS deployment:

	RDCB – Remote Desktop Connection Broker. This is the “hub” of the RDS environment. It ensures that all user connections that are established to the various Session Hosts are maintained through disconnects and reconnects and play a key role in simplifying the single sign on experience
	RDWA – Remote Desktop Web Access. A web site that simply hosts the list of available resources that can be reached through RDS. It also hosts an RSS feed that can be used in various places.
	RDSH – Remote Desktop Session Host. The server that actually runs the user processes. This is what people sometimes refer to as a Terminal Server, although that term has officially been depreciated. When a user runs a RemoteApp or connects to a Desktop, it’s running on a Session Host.

In addition to those three, there are a couple other roles that you can deploy to add more functionality:

	RDGW – Remote Desktop Gateway. Another web site that is actually used as a way of tunneling RDP traffic over HTTPS to allow users who are outside the corporate network to gain access to internal resources. I usually like to co-locate this role on the RDWA server, and I end up referring to RDGW as the “Gateway and Web server”.
	RDVH – Remote Desktop Virtualization Host. A new role for Windows Server 2012, this is a physical server running Hyper-V and is used to deploy and manage Virtual Machines for VDI.
	RDLI – Remote Desktop Licensing. Installing RDS will give you 120 days to try it out, but if you decide to keep it you’ll need to get licensing from Microsoft, and the license key gets installed on the RDLI server. I usually like to co-locate this role on the RDCB.

Many of these roles can be co-located so you can have one server operating many of the roles, or you can deploy a new server for each one. The only role that requires a physical server is the RDVH because that is a Hyper-V Host. Personally I like to start out a deployment with three Virtual Machines:

1. A Connection Broker and License Server
2. A Gateway and Web Access server
3. A Session Host / RemoteApp server

A deployment like that can be easily expanded to fit the needs of the business, like making the roles highly available or adding on a VDI deployment.

I’ll show you how you can set up RemoteApp publishing environment using three servers in about as much time as it takes to do it watch an episode of Dexter! As crazy as it might sound, setting up a three server environment really doesn’t take much longer than a single server deployment but it offers you some fantastic flexibility and growth options. There two things that you’ll need in place before you start your timer:

1. RDS in Windows Server 2012 requires Active Directory.
2. Three servers running a fresh and updated install of Windows Server 2012 joined to that domain.

Once you’ve got that, break out your stop watch. I’ll race you! We’ll be breaking out the three fundamental RDS roles, one on each server:

RDCB	RDWA	RDSH
Remote Desktop Connection Broker.This is the “hub” of the RDS environment. It ensures that all user connections that are established to the various Session Hosts are maintained through disconnects and reconnects and play a key role in simplifying the single sign on experience.	Remote Desktop Web Access. A web site that simply hosts the list of available resources that can be reached through RDS. It also hosts an RSS feed that can be used in various places.	Remote Desktop Session Host. The server that actually runs the user processes. This is what people sometimes refer to as a Terminal Server, although that term has officially been depreciated. When a user runs a RemoteApp or connects to a Desktop, it’s running on a Session Host.

One of the great new features of the new Server Manager is that you can mange multiple servers from the one console. There is no better example of the power that this offers than in deploying and managing Remote Desktop Services. From the new Server Manager, click the Manage menu and select Add Servers. Search for your three servers that will be used for RDS and add them to the selected list by using the right arrow button. Once they’ve been added to the Server Manager, click on the Manage menu and select Add Roles and Features. In addition to being able to manage more than one server now, the new Server Manager also introduces scenario-based installation. Remote Desktop Services is the only “scenario” installation type that is available, but that’s exactly what we want to do. In order to use more than one server for RDS, we’ll do a Standard deployment. The Virtual Desktop Infrastructure (VDI) scenario will be used to allow each user to have their very own virtual machine, but we want to deploy the Session Virtualization scenario which is analogous to what everyone thinks of with Terminal Services; multiple user sessions working independently on one server. The next screen will just explain the various roles that will be deployed by using this wizard. First we’ll select the Connection Broker Then the Web Access server. Notice that you are given the option to install the RDWA on the Connection Broker server. This would allow you to do a Standard deployment with as few as two servers, but I prefer to leave the RDCB and RDWA on their own servers and later deploy the Gateway role to the same server running RDWA. And finally we’ll select the Session Host server. On the Confirmation page you’ll have to check the “Restart” option as the installation of the Session Host role requires a reboot. Then click Deploy. After the roles are deployed and the session host reboots, the Server Manager should show you the status: Succeeded! After clicking Close, you’ll see a new “Remote Desktop Services” page on the left. Select that then click on Collections. “Collections” is a new term that describes a set of services that the RDS deployment offers such as a collection of RemoteApps, Desktop Sessions or Virtual Desktops. From the Tasks button, select Create Session Collection. Enter a Collection Name, something clever like RemoteApps works well. Now select your Session Host server and click the arrow to add it to the Selected list. There should only be the one server available here so it’s pretty straight forward. The default group of users that are allowed to access the applications in this collection will be Domain Users. You can be more specific if you wish, but you can also be more specific on an individual application bases as you publish them later. To keep things moving quickly, let’s skip the User Profile Disks for now. This is a very cool new feature of Windows Server 2012 (8 beta) that allows users on the session host to have their “local” data get automatically redirected to a different virtual hard drive instead of getting written to the actual session host server, but you can configure that later. Click Next then Create to finish the Collection wizard. When it’s done, you can click Close. Now it’s time to publish the applications you really want to give users access to. From the Remote Desktop Services page, select the new RemoteApps collection you made and then from the Tasks button by RemoteApp Programs, select Publish RemoteApp Programs. You can select a program from the list or click “Add Another Program” to browse to an executable. When you’re happy with your selection click Publish, then Close. And that really completes the set up the Standard deployment. You now have a Web Access, Broker and Session Host deployed with applications published via RemoteApp. Way to go you! So how to you test it out? If you want to test it from one of your new servers, let’s first, let’s turn off the IEESC. From the Server Manager, select the Local Server page and click the link next to IE Enhanced Security Configuration and set it to Off. Now open Internet Explorer (run c:program files (x86)internet exploreriexplore.exe) and enter the HTTPS url for your RDWA server, appending /rdweb to the hostname. For this example… https://rede-rdgw-01.techrede.net/rdweb This can be made easier to remember for you users by creating a DNS alias (CNAME) and even set up HTTP redirection later on. After passing the certificate warning you’ll be promoted to run an ActiveX Control. Allow that to run and then log in. Once connected you should see your custom list of applications that are available, so click on one of them to launch the RemoteApp. You’ll be prompted by Internet Explorer with a warning that the Web Site is trying to start a program on your computer. It’s using the Active-X Control to launch the local RDP client (mstsc.exe). This warning can be suppressed by Group Policy once the web site certificate is replaced, but for now just click Connect. Once connected, the application would look just like any locally installed application, but you’ll notice a new system tray icon will show that you are connected to a Remote Work Place. And there you have it, RDS, Quick and Easy on three servers in about an hour. Now you can install new applications and publish them to your Collection. Just like Windows 2008 R2, you can deliver these RemoteApps from RDWeb or by subscribing to the RemoteApp RSS feed. If you want to make these applications available outside of your organization, the next step will be to deploy the RD Gateway role, or if you want to go bigger, try adding more Session Hosts, the equivalent to a RDS Farm

As the name implies, Remote Desktop Services is a way of delivering services for desktops that are not “local”. However, the Quick and Standard deployments of RDS do not include a key component that makes these services available from outside your organization: the RDS Gateway. This role is acts at a proxy over HTTPS to allow a client to tunnel over SSL to your internal resources, limiting exposure and securing communications. In Server Manager, if you want to deploy a separate server for the RDGW role, you’ll want to add that new server to the console which is already managing the rest of your RDS environment. I like to use the manager on the RDCB for this, but any Server Manager console that is managing all of your RDS hosts will work just the same.

In this example I am going to be adding the role to the same server that is already running the RDWA role, so the RDGW and RDWA will be on one server. From the Remote Desktop Servcies area just click on the big green + above RD Gateway to get started.

Select the server that you want to install the role and add it to the Selected list on the right.Pick a DNS name that clients will connect to in order to use the Gateway

This should be the External DNS name that can be resolved to an IP address that will NAT port 443 to the RDGW server. NOTE: In this example the RDGW and RDGA roles are on the same server, both of which use port 443. However, if you also NAT port 80 then the RDWA server will redirect web browsers from HTTP to HTTPS. Without access to port 80 your users will have to remember to type https:// when accessing the RDWA. It’s just being nice to your users really. Also notice that the wizard mentions a Self-Signed Certificate. We will change this in just a moment, so click Next.

On the Confirmation page just click Add if you’re happy with the config.

Once completed successfully click Close.

Notice the warning that a certificate must be configured. You can click on Configure certificate, but if you click Close you can still manage the certificate by selecting “Edit Deployment Properties” under the Overview Tasks.

At this point you can decide to create a new Self-signed certificate that you would apply to all roles or if you’re going to be putting this into production I would suggest that you should be using a 3rd party certificate that all clients will trust be default. I prefer a wildcard certificate for the external domain name being used for the RDWA and RDGW roles.

When you click “Select existing certificate” you will want to select a .pfx file that contains the Private Key of the certificate. Without the Private Key, the server will not be able to use the certificate. Once you’ve entered the password and checked the box to allow it to be added to the trust root CAs, click OK and then Apply the changes.Once you apply the certificate, do it again for all the remaining roles.

Now your client computers can use the Gateway setting found under More Options / Advanced / Connect from anywhere Settings. Under Server Name simply punch in the external FQDN of the gateway server.

With that set you can now try connecting to the internal name of any server on your company network. When you are prompted for credentials you’ll notice the broker name is listed as one of the servers in the connection path.

And you’re all set! Now you can use RemoteApp and Desktops from anywhere

It’s not talked about very much but there is a somewhat hidden feature in Windows Server 2012 Remote Desktop Services that allows you to include a Windows Server 2008 R2 Session Host in the 2012 environment, and it’s really easy! Why would you do this? Why would you want to have a mix of Server 2012 and Server 2008 R2? Perhaps you want to do migrate from an existing 2008 R2 RDS environment rather than do a complete cut over. Or perhaps you find an application that requires Server 2008 R2 for support reasons. Or maybe you just want to publish an older version of Internet Explorer (you could publish IE8 or IE9 this way). Whatever the reason, Microsoft has included a line in the web.config file of the RDWeb server that allows you to access applications (or the desktop) on a 2008 R2 RDSH server. Nothing to install nor even a service to restart. Just edit the file, list the server and poof, your done! In case you haven’t set up a 2008 R2 session host before, I’ll show you how to do that and how to configure it to allow the 2012 RDWA server to read the list of available applications. Let’s get started.

Set up the 2008 R2 Session Host

Build yourself a server with Window Server 2008 R2 SP1 and join your domain. then open the Server Manager and start the Add Roles wizard. We want to set up Remote Desktop Services And select the Remote Desktop Session Host All the other role services will be managed by the Windows 2012 servers. You can pick “Do not require NLA” if you want to allow older clients like XP to connect to RDS, but if you’re going to be a Windows 7 and Windows 8 shop, you may want to consider using NLA to provide better security. You can worry about licensing later, or if you’ve already added a RDLI server you may want to select the mode now. Per User is most common. Select the group of users that should have access to this RDS Server. Note: this should be a pretty generic group, like Domain Users or a smaller list of users that you allow to access RDS. You can later get more granular about who has access to specific applications. This group determines who can access the environment as a whole. If you want to allow this Session Host to play audio and video, then check the first box. this will install the “Desktop Experience” feature which includes a set of codecs, audio support and other apps that are typically found only on desktop operating systems (like the clipping tool, etc). You may want to include the Desktop Composition option as well if you want to “chrome” on RemoteApps to look more like a Windows Client application would (the Aero curved corners and transparency) . Review your selections and click Install. When it’s finished you’ll need to reboot. After the restart, make sure you log in as the same user that initially installed the Role to complete the installation. If you have already assigned certificates to be used on your Server 2012 environment then you should import that certificate on the 2008 R2 server now. Just open the Certificates MMC for the local computer account and import it to the Personal store. Make sure you have the Private Key as well! From the Server Manager, open the Local Users and Groups and double click the “TW Web Access Computers” group. You want to add the Computer account of your Server 2012 RDWA server. In my example, the RDGW and RDWA roles are on the same computer, thus the name in the screen shot is my “Gateway and Web” server. Under Roles, select the RD Session Host Configuration and double click the RDP-Tcp connection. Here you can select the Certificate you imported earlier. From the RemoteApp Manager, click the Change link near “Digital Signature Settings” to select the certificate that should be used to sign the RemoteApp files. If you have a windows 2012 Remote Desktop Gateway server, then you should also click the Change link for RD Gateway Settings and enter the external FQDN of the Gateway as this values gets written into the RDP files so external users can connect to the session host.

Publish an Application

Now it’s time to publish an app! From the RemoteApp Manager, right click in the RemoteApp Programs area and select Add RemoteApp Programs. You can either select an app that’s already listed, but for this example I want to publish Internet Explorer. So click Browse and navigate to the location of the EXE file. You may find that the location gets sent to a UNC path, just make sure you set the local drive path. In this particular case I also want to make sure I publish the 32-bit version, and I also like to set the Name and Alias to something more friendly like “IE9”. I’ve also found that sometimes the icon does not appear in RDWeb when using the x86 path to program files, so picking from the x64 exe seems to correct that. And you’re done! This Windows Server 2008 R2 session host is ready to go.

Set up the 2012 Web Access server

And now for the easy part… On your Windows Server 2012 RDWA server, openC:WindowsWebRDWebWeb.config in notepad and search for “ws2008r2rdserver” and set the value to the FFQDN of your 2008 R2 RDSH. When you try to save the file you might not be able to write it because it’s under the Windows folder and it’s protected by UAC, so save it to your desktop and then copy it from there back to the RDWeb folder. You’ll be prompted for admin rights to do it, and to overwrite the original file. And you’re done! If you load up your RDWeb page you should see your new application listed right along with all the other apps that are on you other 2012 Session Hosts. Neat trick, thanks Microsoft!

IT Landscape for sysadmins

Protocols

IMAP/POP3

Dovecot ( http://www.dovecot.org/ )

IMAP and POP3 server written primarily with security in mind.

Cyrus ( http://cyrusimap.org/ )

Intended to be run on sealed servers, where normal users are not permitted to log in.

DBMail ( https://github.com/pjstevns/dbmail )

Fast and scalable email services, storage of mail messages in a relational database.

Qpopper ( http://www.eudora.com/products/unsupported/qpopper/ )

One of the oldest and most popular server implementations of POP3.

Courier ( http://www.courier-mta.org/imap/ )

Fast, scalable, enterprise IMAP and POP3 server.

HTTP

Nginx ( http://nginx.org/ )

Reverse proxy, load balancer, HTTP cache, and web server.

GWAN ( http://gwan.com )

Tiny, fast & efficient web & app server. All in one 300k executable.

Varnish ( https://www.varnish-cache.org/ )

HTTP based web application accelerator focusing on optimizing caching and compression.

Caddy ( https://caddyserver.com/ )

Caddy is a lightweight, general-purpose web server.

Apache ( http://httpd.apache.org/ )

Most popular web server.

Cherokee ( http://cherokee-project.com/ )

Lightweight, high-performance web server/reverse proxy.

HAProxy ( http://www.haproxy.org/ )

Software based load Balancing, SSL offloading and performance optimization, compression, and general web routing.

uWSGI ( https://github.com/unbit/uwsgi/ )

The uWSGI project aims at developing a full stack for building hosting services.

Tomcat ( http://tomcat.apache.org )

Apache Tomcat is an open source software implementation of the Java Servlet and JavaServer Pages technologies.

Lighttpd ( http://www.lighttpd.net/ )

Web server more optimized for speed-critical environments.

SMTP

Maildrop ( https://github.com/m242/maildrop )

Open Source disposable email SMTP server, also useful for development.

MailHog ( https://github.com/mailhog/MailHog )

Inspired by MailCatcher written in Go, SMTP MTA, web UI and retrieve them with the JSON API.

Haraka ( http://haraka.github.io/ )

A high-performance, pluginable SMTP server written in JavaScript.

Postfix ( http://www.postfix.org/ )

Fast, easy to administer, and secure Sendmail replacement.

iRedMail ( http://www.iredmail.org/ )

An open source mailserver solution.

MailDev ( http://djfarrelly.github.io/MailDev/ )

SMTP Server + Web Interface for viewing and testing emails during development.

OpenSMTPD ( https://opensmtpd.org/ )

Secure SMTP server implementation from the OpenBSD project.

Exim ( http://www.exim.org/ )

Message transfer agent (MTA) developed at the University of Cambridge.

Qmail ( http://cr.yp.to/qmail.html )

Secure Sendmail replacement.

MailCatcher ( http://mailcatcher.me/ )

Ruby gem that deploys a simply SMTP MTA gateway that accepts all mail and displays in web interface. Useful for debugging or development.

Mailcow ( http://mailcow.email/ )

Mailcow is a mailserver suite.

Sendmail ( http://www.sendmail.com/sm/open_source/ )

Message transfer agent (MTA).

VBoxAdm ( http://www.vboxadm.net/ )

Web based GUI for E-Mail servers like Postfix and Dovecot.

hMailServer ( https://www.hmailserver.com )

hMailServer is a free, open source, e-mail server for Microsoft Windows.

DNS

PowerGate ( https://github.com/bobsta63/powergate )

PowerGate is a simple web application built for managing PowerDNS records.

PowerDNS ( https://www.powerdns.com/ )

DNS server with a variety of data storage back-ends and load balancing features.

Designate ( https://wiki.openstack.org/wiki/Designate )

DNS REST API that support several DNS servers as its backend.

NSD ( http://www.nlnetlabs.nl/projects/nsd/ )

Authoritative only, high performance, simple name server.

Bind ( https://www.isc.org/downloads/bind/ )

The most widely used name server software.

Knot ( https://www.knot-dns.cz/ )

High performance authoritative-only DNS server.

dnsmasq ( http://www.thekelleys.org.uk/dnsmasq/doc.html )

A lightweight service providing DNS, DHCP and TFTP services to small-scale networks.

Unbound ( http://unbound.net/ )

Validating, recursive, and caching DNS resolver.

Yadifa ( http://yadifa.eu/ )

Lightweight authoritative Name Server with DNSSEC capabilities powering the .eu top-level domain.

djbdns ( http://cr.yp.to/djbdns.html )

A collection of DNS applications, including tinydns.

TinyDNS ( https://cr.yp.to/djbdns/tinydns.html )

TinyDNS is an lightweight nameserver which is easy to setup.

LDAP

Fusion Directory ( http://www.fusiondirectory.org )

Improve the Management of the services and the company directory based on OpenLDAP.

pGina ( http://pgina.org/ )

Pluggable credential Provider.

OpenLDAP ( http://openldap.org/ )

Developed by the OpenLDAP Project.

FreeIPA ( http://www.freeipa.org )

FreeIPA is an integrated security information management solution combining Linux (Fedora), 389 Directory Server, MIT Kerberos, NTP, DNS, Dogtag (Certificate System).

OpenDJ ( http://opendj.forgerock.org/ )

Fork of OpenDS.

389 Directory Server ( http://port389.org )

Developed by Red Hat.

OpenDS ( https://opends.java.net/ )

Another directory server written in Java.

Apache Directory Server ( http://directory.apache.org/ )

Apache Software Foundation project written in Java.

SSH

autossh ( http://www.harding.motd.ca/autossh/ )

Automatically respawn ssh session after network interruption.

ssh-cert-authority ( https://github.com/cloudtools/ssh-cert-authority )

A democratic SSH certificate authority.

DSH ( http://www.netfort.gr.jp/~dancer/software/dsh.html.en )

Dancer’s shell / distributed shell – Wrapper for executing multiple remote shell commands from one command line.

sshmux ( https://github.com/joushou/sshmux )

SSH multiplexing library, allowing you to write "jump host" style proxies.

Clustershell ( http://cea-hpc.github.io/clustershell/ )

Run commands on multiple hosts in parallel. Clustershell can operate on predefined groups of hosts.

Mosh ( http://mosh.mit.edu/ )

The mobile shell.

putty ( http://www.chiark.greenend.org.uk/~sgtatham/putty/ )

Free and open source terminal emulator.

Dropbear ( https://matt.ucc.asn.au/dropbear/dropbear.html )

Dropbear is a relatively small SSH server and client. It runs on a variety of POSIX-based platforms.NEW

VPN

FreeLan ( https://github.com/freelan-developers/freelan )

Full-mesh, secure, easy-to-setup, multi-platform, open-source, highly-configurable VPN software.

SoftEther ( https://www.softether.org/ )

Multi-protocol software VPN with advanced features.

Pritunl ( http://pritunl.com/ )

OpenVPN based solution. Easy to set up.

tinc ( http://www.tinc-vpn.org/ )

Distributed p2p VPN.

PeerVPN ( https://github.com/peervpn/peervpn )

Virtual network built by PeerVPN uses a full mesh topology.

strongSwan ( http://www.strongswan.org/ )

Complete IPsec implementation for Linux.

OpenVPN ( https://community.openvpn.net )

Uses a custom security protocol that utilizes SSL/TLS for key exchange.

Cloud and Virtualization

Cloud Computing

OpenNode ( http://opennodecloud.com )

Builds open-source infrastructure management software and implements cloud systems.

The Foreman ( http://theforeman.org/ )

Foreman is a complete lifecycle management tool for physical and virtual servers. FOSS.

OpenStack ( https://www.openstack.org/ )

Open source software for building private and public clouds.

Tsuru ( https://tsuru.io )

Tsuru is an extensible and open source Platform as a Service software.

OpenNebula ( http://opennebula.org/ )

An user-driven cloud management platform for sysadmins and devops.

CoreOS ( https://coreos.com/ )

Open Source Projects for Linux Containers

Cracow Cloud One ( https://github.com/cc1-cloud/cc1/ )

The CC1 system provides a complete solution for Private Cloud Computing.

Flynn ( https://flynn.io )

Open source PaaS

Archipel ( http://archipelproject.org/ )

Manage and supervise virtual machines using Libvirt.

Mesos ( http://mesos.apache.org/ )

Develop and run resource-efficient distributed systems.

Cobbler ( http://www.cobblerd.org/ )

Cobbler is a Linux installation server that allows for rapid setup of network installation environments.

OpenShift ( http://www.openshift.org )

OpenShift is a platform as a service product from Red Hat.

CloudStack ( http://cloudstack.apache.org/ )

Cloud computing software for creating, managing, and deploying infrastructure cloud services.

Project-FiFo ( https://project-fifo.net )

Open source SmartOS cloud management with a focus on availability.

AppScale ( http://github.com/AppScale/appscale )

Open source cloud software with Google App Engine compatibility.

Eucalyptus ( https://www.eucalyptus.com/ )

Open source private cloud software with AWS compatibility.

Cloud Orchestration

Mina ( http://nadarei.co/mina/ )

Really fast deployer and server automation tool (rake based).

Overcast ( http://andrewchilds.github.io/overcast/ )

Deploy VMs across different cloud providers, and run commands and scripts across any or all of them in parallel via SSH.

Rundeck ( http://rundeck.org/ )

Simple orchestration tool.

Rocketeer ( http://rocketeer.autopergamene.eu/ )

PHP task runner and deployment tool.

SaltStack ( http://www.saltstack.com/ )

Extremely fast and scalable systems and configuration management software.

BOSH ( http://docs.cloudfoundry.org/bosh/ )

IaaS orchestration platform originally written for deploying and managing Cloud Foundry PaaS, but also useful for general purpose distributed systems.

Juju ( https://juju.ubuntu.com/ )

Cloud orechestration tool which manages services as charms, YAML configuration and deployment script bundles.

MCollective ( http://puppetlabs.com/mcollective )

Ruby framework to manage server orchestration, developed by Puppet labs.

Cloudify ( http://www.getcloudify.org/ )

Open source TOSCA-based cloud orchestration software platform written in Python and YAML.

StackStorm ( http://stackstorm.com/ )

Event Driven Operations and ChatOps platform for infrastructure management. Written in Python.

Marathon ( https://mesosphere.github.io/marathon/ )

A cluster-wide init and control system for services in cgroups or Docker containers.

CloudSlang ( http://www.cloudslang.io )

Flow-based orchestration tool for managing deployed applications, with Docker capabilities.

Capistrano ( http://www.capistranorb.com/ )

Deploy your application to any number of machines simultaneously, in sequence or as a rolling set via SSH (rake based).

Xen Orchestra ( https://xen-orchestra.com )

Xen Orchestra offers a powerful web UI for controlling a complete XenServer or Xen infrastructure.

Cloud Storage

sandstorm ( https://github.com/sandstorm-io/sandstorm )

Personal Cloud Sandbox, install apps to create documents, spreadsheets, blogs, git repos, task lists and more.

Syncthing ( http://syncthing.net/ )

Open Source system for private, encrypted and authenticated distrobution of data.

Nextcloud ( https://nextcloud.com/ )

Next cloud is a fork of OwnCloud.

Pydio ( https://pyd.io )

Pydio is a mature open source software solution for file sharing and synchronization.

ownCloud ( https://owncloud.org )

Provides universal access to your files via the web, your computer or your mobile devices.

Seafile ( http://seafile.com )

Another Open Source Cloud Storage solution.

Swift ( http://docs.openstack.org/developer/swift/ )

A highly available, distributed, eventually consistent object/blob store.

git-annex assistant ( http://git-annex.branchable.com/assistant/ )

A synchronised folder on each of your OSX and Linux computers, Android devices, removable drives, NAS appliances, and cloud services.

SparkleShare ( http://sparkleshare.org/ )

Provides cloud storage and file synchronization services. By default, it uses Git as a storage backend.

Virtualization

Ganeti ( https://code.google.com/p/ganeti/ )

Cluster virtual server management software tool built on top of KVM and Xen.

Proxmox ( http://pve.proxmox.com/wiki/Main_Page )

Open Source Server Virtualization Platform, based on KVM and OpenVZ.

Xen ( http://www.xenproject.org/ )

Virtual machine monitor for 32/64 bit Intel / AMD (IA 64) and PowerPC 970 architectures.

Packer ( http://www.packer.io/ )

A tool for creating identical machine images for multiple platforms from a single source configuration.

oVirt ( http://www.ovirt.org/ )

Manages virtual machines, storage and virtual networks.

LXC – Linux Containers ( https://linuxcontainers.org/ )

System containers which offer an environment as close to possible as the one you’d get from a VM, but without the overhead that comes with running a separate kernel and simulating all the hardware.

rkt ( https://coreos.com/rkt/docs/latest/ )

A fast, composable, and secure App Container runtime for Linux.

KVM ( http://www.linux-kvm.org )

Linux kernel virtualization infrastructure.

Vagrant ( https://www.vagrantup.com/ )

Tool for building complete development environments.

VirtualBox ( https://www.virtualbox.org/ )

Virtualization product from Oracle Corporation.

Software Containers

SmartOS ( https://smartos.org/ )

SmartOS is a hypervisor.

Flocker ( https://github.com/ClusterHQ/flocker )

Flocker is an open-source Container Data Volume Manager for your Dockerized applications.

Rancher ( http://rancher.com/ )

A complete infrastructure platform for running Docker in production.

Kubernetes ( http://kubernetes.io/ )

Kubernetes is an open source orchestration system for Docker containers.

Docker ( http://www.docker.com/ )

Open platform for developers and sysadmins to build, ship, and run distributed applications.

OpenVZ ( http://openvz.org )

Container-based virtualization for Linux.

ElasticKube ( https://elastickube.com )

Enterprise container management for Kubernetes

Messaging

Log Management

Octopussy ( http://www.octopussy.pm )

Log Management Solution (Visualize / Alert / Report).

Kibana ( http://www.elasticsearch.org/overview/kibana/ )

Visualize logs and time-stamped data.

Flume ( https://flume.apache.org/ )

Distributed log collection and aggregation system.

Graylog2 ( http://graylog2.org/ )

Pluggable Log and Event Analysis Server with Alerting options.

Fluentd ( http://www.fluentd.org/ )

Log Collector and Shipper.

Heka ( http://hekad.readthedocs.org/en/latest/ )

Stream processing system which may be used for log aggregation.

Logstash ( http://logstash.net/ )

Tool for managing events and logs.

Solr ( http://lucene.apache.org/solr/ )

Solr is fast search platform built on Apache Lucene.

ElasticSearch ( http://www.elasticsearch.org/ )

A Lucene Based Document store mainly used for log indexing, storage and analysis.

Queuing

Gearman ( http://gearman.org/ )

Fast multi-language queuing/job processing platform.

ZeroMQ ( http://zeromq.org/ )

Lightweight queuing system.

RabbitMQ ( http://www.rabbitmq.com/ )

Robust, fully featured, cross distro queuing system.

NSQ ( http://nsq.io/ )

A realtime distributed messaging platform.

BeanstalkD ( http://kr.github.io/beanstalkd/ )

A simple, fast work queue.

Apache Kafka ( http://kafka.apache.org )

A high-throughput distributed messaging system.

ActiveMQ ( https://activemq.apache.org )

Open source java message broker

Sidekiq ( https://github.com/mperham/sidekiq )

Simple, efficient background processing for Ruby.

QPid ( https://qpid.apache.org/ )

Apache QPid, open source AMQP 1.0 Server and framework.

Memcached ( https://memcached.org/ )

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.NEW

Storage

Cloning

Redo Backup ( http://redobackup.org/ )

Easy Backup, Recovery and Restore.

Fog ( http://www.fogproject.org/ )

Another computer cloning solution.

Clonezilla ( http://clonezilla.org/ )

Partition and disk imaging/cloning program.

OPSI ( http://www.opsi.org )

OPSI is an open source Client Management System for Windows clients and is based on Linux servers

Backups

Elkarbackup ( https://github.com/elkarbackup/elkarbackup )

Backup solution based on RSnapshot with a simple web interface.

ZBackup ( http://zbackup.org/ )

A versatile deduplicating backup tool

Burp ( http://burp.grke.org/ )

Network backup and restore program.

Obnam ( http://obnam.org/ )

Network backup and restore, with snapshotting, deduplication and encryption.

Backupninja ( https://labs.riseup.net/code/projects/backupninja )

Lightweight, extensible meta-backup system.

Bareos ( https://www.bareos.org/ )

A fork of Bacula backup tool.

BorgBackup ( https://borgbackup.readthedocs.org/en/stable/# )

BorgBackup is a deduplicating backup program.

Yadis! Backup ( http://www.codessentials.com/ )

Yadis! Backup is a real time Backup application.

UrBackup ( http://www.urbackup.org/ )

Another client-server backup system.

Lsyncd ( https://github.com/axkibe/lsyncd )

Watches a local directory trees for changes, and then spawns a process to synchronize the changes. Uses rsync by default.

Amanda ( http://www.amanda.org/ )

Client-server model backup tool.

Bacula ( http://www.bacula.org )

Another Client-server model backup tool.

Rsnapshot ( http://www.rsnapshot.org/ )

Filesystem Snapshotting Utility.

Backuppc ( http://backuppc.sourceforge.net/ )

Client-server model backup tool with file pooling scheme.

Duplicity ( http://duplicity.nongnu.org/ )

Encrypted bandwidth-efficient backup using the rsync algorithm.

Snebu ( http://www.snebu.com )

Snebu is an efficient incremental snapshot style client/server disk-based backup system for Unix / Linux systems.

Attic ( https://attic-backup.org )

Attic is a deduplicating backup program written in Python.

Relax and Recover ( http://relax-and-recover.org/ )

Bare metal backup software.

Cobian Backup ( http://www.cobiansoft.com/cobianbackup.htm )

Cobian Backup is a easy Backup software.

SafeKeep ( http://safekeep.sourceforge.net/ )

Centralized pull-based backup using rdiff-backup.

Duplicati ( http://www.duplicati.com )

Duplicati is a backup client that securely stores encrypted, incremental, compressed backups on cloud storage services and remote file servers.

Duply ( http://duply.net/ )

Duply is a frontend for duplicity, which is python based shell application that makes encrypted incremental backups to remote storages.

rdiff-backup ( http://www.nongnu.org/rdiff-backup/ )

Incremental File Backup software.

Deja Dup ( https://launchpad.net/deja-dup )

Deja Dup is a simple backup tool with GUI.

Box Backup ( https://www.boxbackup.org/ )

Is another backup system.

kvmBackup ( https://github.com/bioinformatics-ptp/kvmBackup )

A software for snapshotting KVM images and backing them up.NEW

Distributed Filesystems

GlusterFS ( http://www.gluster.org/ )

Scale-out network-attached storage file system.

XtreemFS ( http://www.xtreemfs.org/ )

XtreemFS is a fault-tolerant distributed file system for all storage needs.

Ceph ( http://ceph.com/ )

Distributed object store and file system.

Sheepdog ( https://sheepdog.github.io/sheepdog/ )

Sheepdog is a distributed object storage system for volume and container services and manages the disks and nodes intelligently.

MooseFS ( http://www.moosefs.org/ )

Fault tolerant, network distributed file system.

DRBD ( http://www.drbd.org/ )

Disributed Replicated Block Device.

LeoFS ( http://leo-project.net )

Unstructured object/data storage and a highly available, distributed, eventually consistent storage system.

MogileFS ( http://mogilefs.org/ )

Application level, network distributed file system.

TahoeLAFS ( https://tahoe-lafs.org/trac/tahoe-lafs )

secure, decentralized, fault-tolerant, peer-to-peer distributed data store and distributed file system.

Lustre ( http://lustre.opensfs.org/ )

A type of parallel distributed file system, generally used for large-scale cluster computing.

OpenAFS ( http://www.openafs.org/ )

Distributed network file system with read-only replicas and multi-OS support.

BeeGFS ( http://www.beegfs.com/content/ )

BeeGFS is the leading parallel cluster file system, developed with a strong focus on performance.

HDFS ( http://hadoop.apache.org/ )

Distributed, scalable, and portable file-system written in Java for the Hadoop framework.

RDBMS

Firebird ( http://www.firebirdsql.org/ )

True universal open source database.

PostgreSQL-XL ( http://www.postgres-xl.org/ )

Scalable Open Source PostgreSQL-based database cluster.

MariaDB ( https://mariadb.org/ )

Community-developed fork of the MySQL.

Galera ( http://galeracluster.com/ )

Galera Cluster for MySQL is an easy-to-use high-availability solution with high system up-time, no data loss, and scalability for future growth.

Crate ( https://crate.io/ )

Another easy to use, fast and scalable database system.

TokuDB ( http://www.tokutek.com/tokudb-for-mysql/ )

TokuDB is an open source, high-performance storage engine for MySQL, MariaDB, and Percona Server that dramatically improves scalability and operational efficiency.

Percona Server ( http://www.percona.com/software )

Enhanced, drop-in MySQL replacement.

MySQL ( http://dev.mysql.com/ )

Most popular RDBMS server.

SQLite ( http://sqlite.org/ )

Library that implements a self-contained, serverless, zero-configuration, transactional SQL DBS.

PostgreSQL ( http://www.postgresql.org/ )

Object-relational database management system (ORDBMS).

NoSQL

Redis ( http://redis.io/ )

Networked, in-memory, key-value data store with optional durability.

LevelDB ( https://code.google.com/p/leveldb/ )

Google’s high performance key/value database.

MongoDB ( http://www.mongodb.org/ )

Another document-oriented database system.

CouchDB ( http://couchdb.apache.org/ )

Ease of use, with multi-master replication document-oriented database system.

Hypertable ( http://hypertable.org/ )

C++ based BigTable-like DBMS, communicates through Thrift and runs either as stand-alone or on distributed FS such as Hadoop.

FlockDB ( https://github.com/twitter/flockdb )

Twitter’s distributed, fault-tolerant graph database.

Apache HBase ( http://hbase.apache.org/ )

Hadoop database, a distributed, big data store.

RavenDB ( http://ravendb.net/ )

Document based database with ACID/Transactional features.

Riak ( http://basho.com/riak/ )

Another fault-tolerant key-value NoSQL database.

Neo4j ( http://www.neo4j.org/ )

Open source graph database.

OrientDB ( http://orientdb.com )

Multi-Model Database, mainly Graph Database.

RethinkDB ( http://www.rethinkdb.com/ )

Open source distributed document store database, focuses on JSON.

Cassandra ( http://cassandra.apache.org/ )

Distributed DBMS designed to handle large amounts of data across many servers.

Monitoring

Statistics

Piwik ( http://piwik.org/ )

Free and open source web analytics application.

GoAccess ( http://goaccess.io/ )

Open source real-time web log analyzer and interactive viewer that runs in a terminal.

Webalizer ( http://www.webalizer.org/ )

Fast, free web server log file analysis program.

Monitoring

Log.io ( http://logio.org/ )

Real-time log monitoring.

Bloonix ( https://bloonix.org )

Bloonix is your next-gen monitoring solution!

Sensu ( http://sensuapp.org/ )

Open source monitoring framework.

Shinken ( http://www.shinken-monitoring.org/ )

Another monitoring framework.

Zabbix ( http://www.zabbix.com/ )

Enterprise-class software for monitoring of networks and applications.

Adagios ( http://adagios.org/ )

Adagios is a web based Nagios configuration interface.

Dash ( https://github.com/afaqurk/linux-dash )

A low-overhead monitoring web dashboard for a GNU/Linux machine.

Alerta ( https://github.com/guardian/alerta )

Distributed, scaleable and flexible monitoring system.

zmon ( https://github.com/zalando/zmon )

ZMON is Zalando’s open-source platform monitoring tool.

Sentry ( https://getsentry.com/ )

Application monitoring, event logging and aggregation.

Icinga ( https://www.icinga.org/ )

Fork of Nagios.

Cacti ( http://www.cacti.net )

Web-based network monitoring and graphing tool.

PHP Server Monitor ( http://sourceforge.net/projects/phpservermon/ )

Open source tool to monitor your servers and websites

netdata ( https://github.com/firehol/netdata/ )

Real-time performance monitoring, done right!

Cabot ( http://cabotapp.com/ )

Monitoring and alerts, similar to PagerDuty.

Riemann ( http://riemann.io/ )

Flexible and fast events processor allowing complex events/metrics analysis.

Flapjack ( http://flapjack.io/ )

Monitoring notification routing & event processing system.

Monit ( http://mmonit.com/monit/#home )

Small Open Source utility for managing and monitoring Unix systems.

Nagios ( http://www.nagios.org/ )

Computer system, network and infrastructure monitoring software application.

Open Monitoring Distribution ( http://omdistro.org/ )

Monitoring solution based on Nagios.

BandwidthD ( http://bandwidthd.sourceforge.net/ )

BandwidthD tracks usage of TCP/IP network subnets and builds html files with graphs to display utilization.

Centreon ( https://www.centreon.com/en/ )

Centreon is real-time IT performance monitoring and diagnostics management tool.

LibreNMS ( https://github.com/librenms/librenms/ )

fork of Observium.

NetXMS ( http://www.netxms.org/ )

NetXMS is an enterprise grade multi-platform open source network management and monitoring system.

Selena ( https://github.com/allegro/selena )

Selena is a tool for monitoring website performance by monitoring response times, response codes and site content.

Monitorix ( http://www.monitorix.org/ )

Monitorix is a lightweight system monitoring tool.

Pandora FMS ( http://pandorafms.com/ )

Flexible Monitoring System, a Nagios and Zabbix alternative

check_mk ( http://mathias-kettner.com/check_mk.html )

Collection of extensions for Nagios.

Zenoss ( http://community.zenoss.org )

Application, server, and network management platform based on Zope.

OpenNMS ( http://www.opennms.org )

OpenNMS is enterprise grade network management application platform.

OpenITCockpit ( http://www.openitcockpit.org/en.html )

openITCockpit is an monitoring suite for the monitoring tool naemon.

MuninMX ( https://www.muninmx.com )

MuninMX is a collector and frontend replacement for the Open Source Munin Monitoring Tool and is compatible with deployed munin-nodes which it also uses.

Thruk ( http://www.thruk.org/ )

Multibackend monitoring webinterface with support for Naemon, Nagios, Icinga and Shinken.

Munin ( http://munin-monitoring.org/ )

Networked resource monitoring tool.

Naemon ( http://www.naemon.org/ )

Network monitoring tool based on the Nagios 4 core with performance enhancements and new features.

EyesOfNetwork ( https://www.eyesofnetwork.com )

EyesOfNetwork is the OpenSource solution combining a pragmatic usage of ITIL processes and a technological interface allowing their workaday application.

Observium ( http://www.observium.org/ )

SNMP monitoring for servers and networking devices. Runs on linux.

CactiEZ ( http://cactiez.cactiusers.org )

Monitoring tools with many features

weathermap ( https://network-weathermap.com/ )

Create your own live network maps from the network statistics you already haveNEW

Metric and Metric Collection

Prometheus ( http://prometheus.io/ )

An open-source service monitoring system and time series database.

Packetbeat ( http://packetbeat.com )

Captures network traffic and displays it in a custom Kibana dashboard for easy viewing.

Stashboard ( http://www.stashboard.org/ )

Status dashboard software.

Dashing ( http://dashing.io/ )

Ruby gem that allows for rapid statistical dashboard development. An all HTML5 approach allows for big screen displays in data centers or conference rooms.

Tessera ( https://github.com/urbanairship/tessera )

Easy to configure dashboard for Graphite

Cachet ( https://cachethq.io/ )

Status page system.

Graphite ( http://graphite.readthedocs.org/en/latest/ )

Open source scaleable graphing server.

Logstalgia ( http://logstalgia.io/ )

A website access visualiziation tool.

Statsd ( https://github.com/etsy/statsd/ )

Application statistic listener.

Collectl ( http://collectl.sourceforge.net/ )

High precision system performance metrics collecting tool.

Grafana ( http://grafana.org/ )

A Graphite & InfluxDB Dashboard and Graph Editor.

Smokeping ( https://oss.oetiker.ch/smokeping/ )

Network analytic tool.

Gource ( http://gource.io/ )

Gource is a software version control visualiziation tool.

Ganglia ( http://ganglia.sourceforge.net/ )

High performance, scalable RRD based monitoring for grids and/or clusters of servers. Compatible with Graphite using a single collection process.

InfluxDB ( http://influxdb.com/ )

Open source distributed time series database with no external dependencies.

Collectd ( http://collectd.org/ )

System statistic collection daemon.

RRDtool ( http://oss.oetiker.ch/rrdtool/ )

Open source industry standard, high performance data logging and graphing system for time series data.

DalmatinerDB ( https://dalmatiner.io )

Fast distributed metric store for high throughput environments.

OpenTSDB ( http://opentsdb.net/ )

Store and server massive amounts of time series data without losing granularity.

KairosDB ( https://code.google.com/p/kairosdb/ )

Fast distributed scalable time series database, fork of OpenTSDB 1.x.

Automation

Configuration Management

Ansible ( http://www.ansibleworks.com/ )

It’s written in Python and manages the nodes over SSH.

Boxstarter ( http://boxstarter.org )

Config management for Windows OS

Fabric ( http://www.fabfile.org/ )

Python library and cli tool for streamlining the use of SSH for application deployment or systems administration tasks.

Pallet ( http://palletops.com/ )

Infrastructure definition, configuration and management via a Clojure DSL.

Puppet ( http://puppetlabs.com/ )

It’s written in Ruby and uses Puppet’s declarative language or a Ruby DSL.

Chef ( http://www.opscode.com/chef/ )

It’s written in Ruby and Erlang and uses a pure-Ruby DSL.

CFEngine ( http://cfengine.com/ )

Lightweight agent system. Configuration state is specified via a declarative language.

Terraform ( http://www.terraform.io )

Terraform provides a common configuration to launch infrastructure from physical and virtual servers to email and DNS providers.

Configuration Management Database

Clusto ( https://github.com/clusto/clusto )

Helps you keep track of your inventory, where it is, how it’s connected, and provides an abstracted interface for interacting with the elements of the infrastructure.

iTop ( http://www.combodo.com/-Overview-.html )

A complete open source, ITIL, web based service management tool.

i-doit ( http://www.i-doit.org/ )

Open Source IT Documentation and CMDB.

Service Discovery

ZooKeeper ( http://zookeeper.apache.org/ )

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.

Doozerd ( https://github.com/ha/doozerd )

Doozer is a highly-available, completely consistent store for small amounts of extremely important data.

Consul ( http://www.consul.io/ )

Consul is a tool for service discovery, monitoring and configuration.

etcd ( https://github.com/coreos/etcd )

A highly-available key value store for shared configuration and service discovery.

Network Configuration Management

Oxidized ( https://github.com/ytti/oxidized )

A modern take on network device configuration monitoring with web interace and GIT storage.

GestióIP ( http://www.gestioip.net/ )

An automated web based IPv4/IPv6 IP Address Management tool.

rConfig ( http://www.rconfig.com/ )

Another network device configuration management tool.

RANCID ( http://www.shrubbery.net/rancid/ )

Monitors network device’s configurarion and maintain history of changes.

trigger ( https://github.com/trigger/trigger )

Robust network automation toolkit written in Python.

PHPIpam ( http://phpipam.net/ )

Light, modern and useful web IP address management application

Netbox ( https://github.com/digitalocean/netbox )

NetBox is an IP address management (IPAM) and data center infrastructure management (DCIM) tool. Initially conceived by the network engineering team at DigitalOcean, NetBox was developed specifically to address the needs of network and infrastructure engineers.

Continuous Integration and Continuous Deployment

Go ( http://www.go.cd/ )

Open source continuous delivery server.

Drone ( https://github.com/drone/drone )

Continuous integration server built on Docker and configured using YAML files.

Jenkins ( http://jenkins-ci.org/ )

An extendable open source continuous integration server.

GitLab CI ( https://www.gitlab.com/gitlab-ci/ )

Based off of ruby. They also provide GitLab, which manages git repositories.

Buildbot ( http://buildbot.net/ )

Python-based toolkit for continuous integration.

Support software

Control Panels

Cockpit Project ( http://cockpit-project.org )

Cockpit makes it easy to administer your GNU/Linux servers via a web browser.

Senatora ( http://www.sentora.org/ )

Sentora provides a robust open-source web hosting control panel for small to medium ISPs.

Ajenti ( http://ajenti.org/ )

Control panel for Linux and BSD.

ZPanel ( http://www.zpanelcp.com/ )

Control panel for Linux, BSD, and Windows.

Lan Management System ( https://github.com/lmsgit/lms )

LAN Management System is a package of applications for managing LAN networks.

Webmin ( http://webmin.com/ )

Webmin is an individual Server Control Panel.

Panamax ( http://panamax.io/ )

An open-source project that makes deploying complex containerized apps as easy as Drag-and-Drop.

EasySCP ( http://www.easyscp.net/ )

EasySCP is another server webinterface.

tipboard ( https://github.com/allegro/tipboard )

Tipboard is a system for creating dashboards, written in JavaScript and Python.

ispmanager ( https://www.ispsystem.com/software/ispmanager-en )

Panel for shared web hosting

VestaCP ( http://www.vestacp.com/ )

Hosting panel for Linux but with Nginx.

PDNS Gui ( https://github.com/odoucet/pdns-gui )

Web based GUI which aids in administering domains and records for the PowerDNS with MySQL backend.

ISPConfig ( http://www.ispconfig.org )

Hosting control panel for Linux.

Calamari ( http://calamari.readthedocs.io/en/latest/ )

Calamari is a management and monitoring service for Ceph, exposing a high level REST API.

NethServer ( http://www.nethserver.org/ )

NethServer is an operating system for Linux enthusiasts, designed for small offices and medium enterprises.

Feathur ( http://feathur.com )

VPS Provisioning and Management Software.

WebVirtMgr ( https://retspen.github.io )

libvirt-based Web interface for managing virtual machines.

Centos Web Panel ( http://centos-webpanel.com/ )

Free CentOS Linux Web Hosting control panel designed for quick and easy management.

Atomia DNS ( http://atomiadns.com/ )

Free and open source DNS management system.

i-MSCP ( https://i-mscp.net/ )

Webhosting control panel.

Poweradmin ( http://www.poweradmin.org/ )

Friendly web-based DNS administration tool for PowerDNS server.

Glances ( https://nicolargo.github.io/glances/ )

Glances is a cross-platform curses-based system monitoring tool written in Python.

Bright Game Panel ( http://www.bgpanel.net/ )

Bright Game Panel is an open source gameserver control panel.

Froxlor ( http://www.froxlor.org/ )

Easy to use panel for Linux with Nginx and PHP-FPM support.

ViMbAdmin ( http://www.vimbadmin.net/ )

Provides a web based virtual mailbox administration system, allowing mail administrators to manage domains, mailboxes and aliases.

Easy-WI ( https://easy-wi.com/ )

Easy WI is a professional gameserver control panel.

Virtualmin ( http://www.virtualmin.com/ )

Control panel for Linux based on webmin.

Postfix Admin ( http://postfixadmin.sourceforge.net/ )

Web interface to manage postfix mailboxes, virtual domains and aliases.

OpenVZ Web Panel ( http://owp.softunity.com.ru/ )

Web panel to control your OpenVZ servers.

WebSVN ( http://www.websvn.info/ )

Opensource web subversion repository browser.

iF.SVNAdmin ( http://svnadmin.insanefactory.com/ )

WebGUI to manage Subversion repositories and User/Group permissions.

Webmails

Modoboa ( http://modoboa.org )

Modoboa is a mail hosting and management platform including a modern and simplified Web User Interface. It provides useful components such as an administration panel or a webmail.

RainLoop ( http://www.rainloop.net )

Very nice webmail with IMAP/SMTP Support and multi accounting.

Mailpile ( https://www.mailpile.is/ )

A modern, fast web-mail client with user-friendly encryption and privacy features.

Citadel ( http://citadel.org )

Citadel is a free, open source groupware system.

Horde ( http://www.horde.org )

Webmail and groupware client.

Roundcube ( http://roundcube.net/ )

Browser-based IMAP client with an application-like user interface.

Newsletters

Servers for Hackers ( http://serversforhackers.com/ )

Newsletter for programmers who find themselves needing to know their way around a server.

LibreMailer ( https://github.com/averna-syd/LibreMailer )

Libre Mailer is a modest and simple web based email marketing application.

Lewsnetter ( https://github.com/bborn/lewsnetter )

E-mail marketing application, includes subscription management, delivery, bounce and complaint notification, templates and some stats.

phpList ( http://www.phplist.com/ )

Newsletter manager written in PHP.

Web Operations Weekly ( http://webopsweekly.com )

A weekly newsletter on Web operations, infrastructure, performance, and tooling, from the browser down to the metal.

DadaMail ( http://dadamailproject.com/ )

Mailing List Manager, written in Perl.

Project Management

OpenProject ( https://www.openproject.org )

Project collaboration with open source.

Taiga ( https://taiga.io/ )

Agile, Free, Open Source Project Management Tool based on the Kanban and Scrum methods.

Phabricator ( http://phabricator.org/ )

Written in PHP.

CaseBox ( https://www.casebox.org )

Manage all your organisation’s information in one system.

kanboard ( http://kanboard.net/ )

Kanboard is a project management software that uses the Kanban methodology.

Wekan ( http://wekan.io/ )

Wekan is an open-source and collaborative kanban board application.

Gogs ( http://gogs.io/ )

Written in Go.

GitBucket ( https://github.com/takezoe/gitbucket )

Clone of GitHub written in Scala.

Restyaboard ( http://restya.com/board/index.html )

Trello like kanban board. Restyaboard is based on Restya platform.

Kallithea ( http://kallithea-scm.org/ )

OpenSource Git and mercurial sources management.

Tuleap ( https://www.tuleap.org/ )

Tuleap, 100% Open Source software development and project management tool

Redmine ( http://www.redmine.org/ )

Written in ruby on rails.

GitLab ( https://www.gitlab.com/ )

Clone of GitHub written in Ruby.

Trac ( http://trac.edgewall.org/ )

Written in python.

Ticketing systems

osTicket ( http://osticket.com/ )

Open source support ticket system.

Cerb ( http://www.cerberusweb.com/ )

A group-based e-mail management project built with a commercial open source license.

Flyspray ( http://flyspray.org )

Web-based bug tracking system written in PHP.

Otrs ( http://www.otrs.com/ )

A free and open-source trouble ticket system software package that a company, organization, or other entity can use to assign tickets to incoming queries and track further communications about them.

Request Tracker ( http://www.bestpractical.com/rt/ )

Ticket-tracking system written in Perl.

MantisBT ( http://www.mantisbt.org/ )

Another web-based bug tracking system.

TheBugGenie ( http://www.thebuggenie.com )

Open source ticket system with extremely complete users rights granularity.

Bugzilla ( http://www.bugzilla.org/ )

General-purpose bugtracker and testing tool originally developed and used by the Mozilla project.

Zammad ( http://zammad.org/ )

Zammad is an web-based ticket solution.NEW

IT Asset Management

Snipe IT ( http://snipeitapp.com/ )

Asset & license management software.

Ralph ( https://github.com/allegro/ralph )

Asset management, DCIM and CMDB system for large Data Centers as well as smaller LAN networks.

RackTables ( http://racktables.org/ )

Datacenter and server room asset management like document hardware assets, network addresses, space in racks, networks configuration.

OCS Inventory NG ( http://www.ocsinventory-ng.org/en/ )

Enables users to inventory their IT assets.

GLPI ( http://www.glpi-project.org/spip.php?lang=en )

Information Resource-Manager with an additional Administration Interface.

Open-AudIT ( http://www.open-audit.org/index.php )

Open-AudIT is an application to tell you exactly what is on your network, how it is configured and when it changes.

FusionInventory ( http://fusioninventory.org/ )

Alternative to the OCS Inventory solution, allow to inventory IT assets.

CMDBuild ( http://www.cmdbuild.org )

CMDBuild is a web environment in which you can configure custom solutions for IT Governance, or more generally for asset management. NEW

Wikis

MoinMoin ( http://moinmo.in/ )

An advanced, easy to use and extensible WikiEngine with a large community of users.

XWiki ( http://www.xwiki.org/xwiki/bin/view/Main/WebHome )

XWiki is a professional wiki that has powerful extensibility features such as scripting in pages, plugins and a highly modular architecture.

TiddlyWiki ( http://tiddlywiki.com )

Complete interactive wiki in JavaScript.

Gollum ( https://github.com/gollum/gollum )

A simple, Git-powered wiki with a sweet API and local frontend.

DokuWiki ( https://www.dokuwiki.org/dokuwiki )

Simple to use and highly versatile wiki that doesn’t require a database.

ikiwiki ( http://ikiwiki.info/ )

A wiki compiler.

TWiki ( http://twiki.org/ )

TWiki – the Open Source Enterprise Wiki and Web Application Platform

Sphinx ( http://www.sphinx-doc.org/ )

Python Documentation Generator

Olelo Wiki ( https://github.com/minad/olelo )

A a wiki that stores pages in a Git repository.

TikiWiki ( https://tiki.org/ )

Free Wiki system.

PmWiki ( http://www.pmwiki.org )

Wiki-based system for collaborative creation and maintenance of websites.

Mediawiki ( http://www.mediawiki.org/wiki/MediaWiki )

Used to power Wikipedia.

Code Review

Review Board ( https://www.reviewboard.org/ )

Available as free software uner the MIT License.

Gerrit ( https://code.google.com/p/gerrit/ )

Based on the Git version control, it facilitates software developers to review modifications to the source code and approve or reject those changes.

Collaborative Software

SOGo ( https://www.sogo.nu/ )

Collaborative software server with a focus on simplicity and scalability.

Mattermost ( http://www.mattermost.org/ )

Mattermost brings all your team communication into one place, making it searchable and accessible anywhere.

Kolab ( https://www.kolab.org )

Another groupware suite.

Horde Groupware ( http://www.horde.org/apps/groupware )

PHP based collaborative software suite that includes email, calendars, wikis, time tracking and file management.

Citadel/UX ( http://www.citadel.org/ )

Collaboration suite (messaging and groupware) that is descended from the Citadel family of programs.

Zimbra ( https://www.zimbra.com/community/ )

Collaborative software suite, that includes an email server and web client.

EGroupware ( http://www.egroupware.org/ )

Groupware software written in PHP.

Tine ( http://tine20.github.io/Tine-2.0-Open-Source-Groupware-and-CRM/ )

Tine is a groupware solution.

Communication

Kandan ( http://getkandan.com/ )

Open source self hosted Chat.

Lets-Chat ( http://sdelements.github.io/lets-chat/ )

A self hosted chat suite written in Node.

Hack.Chat ( https://hack.chat/ )

Chat for hacker from hackers

Openfire ( http://www.igniterealtime.org/projects/openfire/ )

Real time collaboration (RTC) server.

Jappix ( https://jappix.org/ )

Web-based chat client

Kaiwa ( http://getkaiwa.com )

Web based chat client in the style of common paid alternatives.

Metronome IM ( http://www.lightwitch.org/metronome )

Fork of Prosody IM.

Discord ( https://discordapp.com/ )

Is a voice and chat software and an alternative to Skype and Teamspeak.

Spreed.Me ( https://www.spreed.me/?far )

Spreed WebRTC implements a WebRTC audio/video call and conferencing server and web client.

Discourse ( http://www.discourse.org/ )

Civilsied discussions.

MongooseIM ( https://www.erlang-solutions.com/products/mongooseim-massively-scalable-ejabberd-platform )

Fork of ejabberd.

HumHub ( https://github.com/humhub/humhub/ )

Social communication network kit.

Rocket.Chat ( https://rocket.chat/ )

Web chat platform.

Tigase ( https://projects.tigase.org/projects/tigase-server )

XMPP server implementation in Java.

Alfresco ( https://www.alfresco.com/community )

The Alfresco Enterprise Content Management platform is an open, powerful ECM platform.

Prosody IM ( http://prosody.im/ )

XMPP server written in Lua.

Candy ( http://candy-chat.github.io/candy )

Multi user XMPP client written in Javascript.

ejabberd ( http://www.ejabberd.im/ )

XMPP instant messaging server written in Erlang/OTP.

Jitsi Meet ( https://jitsi.org/Projects/JitsiMeet )

Jitsi Meet is an OpenSource (MIT) WebRTC JavaScript application that uses Jitsi Videobridge to provide high quality, scalable video conferences.

Asterisk ( http://www.asterisk.org )

Asterisk is a Framework to build Telephony Servers, IVR, Voicemail and more

Matterm ( https://www.mattermost.org )

Open source Slack alternative.

Matrix ( http://matrix.org/ )

Matrix is an open standard for decentralised communication.NEW

Essentials

Editors

neovim ( http://neovim.org/ )

The Vim text editor has been loved by a generation of users. This is the next generation.

Lime ( http://limetext.org/ )

Aims to provide an open source solution to Sublime Text.

ICEcoder ( http://icecoder.net )

Code editor awesomeness, built with common web languages.

Light Table ( http://www.lighttable.com/ )

The next generation code editor.

Atom ( https://atom.io/ )

A hackable text editor from Github.

Haroopad ( http://pad.haroopress.com/ )

Markdown editor with live preview.

TextMate ( https://github.com/textmate/textmate/ )

A graphical text editor for OS X.

Vim ( http://www.vim.org )

A highly configurable text editor built to enable efficient editing.

Brackets ( http://brackets.io/ )

Open source code editor for web designers and front-end developers.

jotgit ( https://github.com/jdleesmiller/jotgit )

Git-backed real-time collaborative code editing.

Notepad++ ( https://notepad-plus-plus.org/ )

Notepad++ is a free source code editor and Notepad replacement that supports several languages.

Geany ( http://www.geany.org/ )

GTK2 text editor.

KDevelop ( https://www.kdevelop.org )

An open source IDE by the people behind KDE.

GNU Emacs ( http://www.gnu.org/software/emacs/ )

An extensible, customizable text editor-and more.

nano ( http://www.nano-editor.org/ )

GNU Nano is a clone of the Pico text editor with some enhancements.

NetBeans IDE ( https://netbeans.org )

NetBeans IDE lets you quickly and easily develop Java desktop, mobile, and web applications, as well as HTML5 applications with HTML, JavaScript, and CSS.

Eclipse ( http://eclipse.org/ )

IDE written in Java with an extensible plug-in system.

Repositories

Dotdeb ( http://www.dotdeb.org/ )

Repository with LAMP updated packages for Debian.

ElRepo ( http://elrepo.org/tiki/tiki-index.php )

The ELRepo Project focuses on hardware related packages to enhance your experience with Enterprise Linux.

Remi ( http://rpms.famillecollet.com/ )

Repository with LAMP updated packages for RHEL/Centos/Fedora.

Pulp ( http://www.pulpproject.org/ )

Pulp is a platform for managing repositories of content, such as software packages, and pushing that content out to large numbers of consumers.

SCM Manager ( https://www.scm-manager.org/ )

The easiest way to share and manage your Git, Mercurial and Subversion repositories.

EPEL ( https://fedoraproject.org/wiki/EPEL )

Repository for RHEL and compatibles (CentOS, Scientific Linux).

Software Collections ( https://www.softwarecollections.org/en/ )

Community Release of Red Hat Software Collections. Provides updated packages of Ruby, Python, etc. for CentOS/Scientific Linux 6.x.

IUS Community Project ( https://iuscommunity.org/pages/Repos.html )

A better way to upgrade RHEL.

RepoForge ( http://repoforge.org/ )

ex-RpmForge

Aptly ( https://www.aptly.info/ )

Mirror, create, snapshot and publish Debian repositories

Security

Bro ( http://www.bro.org )

Bro is a powerful framework for network analysis and security monitoring.

BlackBox ( https://github.com/StackExchange/blackbox )

Safely store secrets in Git/Mercurial. Privides tooling to automatically encrypt secrets like passwords.

Pass ( http://www.passwordstore.org/ )

Password manager based around gpg and git, with a bit of setup allowing easy collaboration.

OpenVAS ( http://www.openvas.org/vm.html )

Open source intrusion detection system

Linux Malware Detect ( https://www.rfxn.com/projects/linux-malware-detect/ )

A malware scanner for Linux designed around the threats faced in shared hosted environments.

LookingGlass ( https://github.com/telephone/LookingGlass )

LookingGlass for hosting on own server.

Vault ( https://www.vaultproject.io )

Vault secures, stores, and tightly controls access to tokens, passwords, certificates, API keys, and other secrets in modern computing.

RatticDB ( http://rattic.org/ )

RatticDB is a password management database.

Suricata ( http://suricata-ids.org/ )

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine.

OsSec ( http://www.ossec.net/ )

Open Source IDS

Vaultier ( https://www.vaultier.org/ )

Collaborative password manager that uses PKI

OPNsense ( https://opnsense.org/ )

OPNsense is an open source, easy-to-use and easy-to-build FreeBSD based firewall and routing platform.

Snort ( https://www.snort.org/ )

Snort is a free and open source network intrusion prevention system and network intrusion detection system.

OpenAS ( https://openas.org/ )

An self-hosted email spam filter.

MailBorder ( http://www.mailborder.com/ )

Free E-Mail Antivirus/Spam Gateway.

Fail2Ban ( http://www.fail2ban.org/wiki/index.php/Main_Page )

Scans log files and takes action on IPs that show malicious behavior.

Kali Linux ( https://www.kali.org/ )

Kali Linux is a Open Source Penetration software.

Rspamd ( https://rspamd.com/ )

Advanced spam filtering system that allows evaluation of messages by a number of rules including regular expressions, statistical analysis and custom services such as URL black lists.

PacketFence ( http://www.packetfence.org )

PacketFence is a fully supported, trusted, Free and Open Source network access control (NAC) solution.

AlienVault OSSIM ( https://www.alienvault.com/products/ossim )

OSSIM provides you with a feature-rich open source SIEM, complete with event collection, normalization and correlation.

WinMTR ( http://winmtr.net/ )

Free Network diagnostic tool.

EFA Project ( https://efa-project.org/ )

Great EMail Filter Appliance.

VyOS ( http://vyos.net/ )

VyOS is a community fork of Vyatta, a Linux-based network operating system that provides software-based network routing, firewall, and VPN functionality.

Untangle ( https://www.untangle.com/ )

Free firewall distribution based on debian with paid addons.

Denyhosts ( http://denyhosts.sourceforge.net/ )

Thwart SSH dictionary based attacks and brute force attacks.

pfSense ( https://www.pfsense.org/ )

Firewall and Router FreeBSD distribution.

Smoothwall ( http://www.smoothwall.org/ )

Software Firewall.

KeePass ( http://keepass.info/ )

Keepass is a great and secure password storage tool.

IPCop ( http://www.ipcop.org/ )

Secure firewall distribution.

OSQuery ( https://osquery.io )

Query your servers status and info using a SQL like interface.

Devil Linux ( http://www.devil-linux.org/home/index.php )

Security distribution which runs from a usb stick or cd.

ClamAV ( http://www.clamav.net/ )

Antivirus software

lcsam ( https://github.com/LiveConfig/lcsam )

Another Spamassassin milter.

rkhunter ( http://rkhunter.sourceforge.net/ )

Rootkit Hunter is a tool that scans for rootkits, backdoors and possible local exploits.

SpamAssassin ( https://spamassassin.apache.org/ )

A powerful and popular email spam filter employing a variety of detection techniques.

chkrootkit ( http://www.chkrootkit.org/ )

chkrootkit is a tool to locally check for signs of a rootkit.

BlackArch ( https://blackarch.org/ )

Is an excellent pentesting distribution.

ipfire ( http://www.ipfire.org/ )

IPFire, open source easy to use firewall.

Passbolt ( https://www.passbolt.com/ )

Secure password database for sharing in a team.NEW

Version control

GitKraken ( http://www.gitkraken.com )

Beautiful cross-platform Git client.

Fossil ( http://www.fossil-scm.org/ )

Distributed version control with built-in wiki and bug tracking.

Git ( http://git-scm.com/ )

Distributed revision control and source code management (SCM) with an emphasis on speed.

Mercurial ( http://mercurial.selenic.com/ )

Another distributed revision control.

GNU Bazaar ( http://bazaar.canonical.com/ )

Distributed revision control system sponsored by Canonical.

Subversion ( http://subversion.apache.org/ )

Client-server revision control system.

Packaging

omnibus-ruby ( https://github.com/opscode/omnibus-ruby )

Full stack, cross distro packaging software (Ruby).

FPM ( https://github.com/jordansissel/fpm )

Easily create any kind of package

Poky ( https://www.yoctoproject.org/tools-resources/projects/poky )

The Yocto Project provides templates, tools and methods to help you create custom Linux-based systems for embedded products.

Open Build Service ( http://openbuildservice.org/ )

A generic system to build and distribute packages from sources in an automatic, consistent and reproducible way.

Troubleshooting

Sysdig ( http://www.sysdig.org/ )

Capture system state and activity from a running Linux instance, then save, filter and analyze.

Wireshark ( http://www.wireshark.org/ )

The world’s foremost network protocol analyzer.

ngrep ( http://ngrep.sourceforge.net/ )

Best tool to grep over the network

Books

UNIX and Linux System Administration Handbook ( http://www.admin.com/ )

Approaches system administration from a practical perspective.

Cybrary ( https://www.cybrary.it/ )

Free security tutorials.

The Linux Command Line ( http://linuxcommand.org/tlcl.php )

A book about the Linux command line by William Shotts.

Debian Administrator HandBook ( https://debian-handbook.info/get/now/ )

This book teaches the essentials to anyone who wants to become an effective and independent Debian GNU/Linux administrator

The Practice of System and Network Administration ( http://everythingsysadmin.com/books.html )

The first and second editions describes the best practices of system and network administration, independent of specific platforms or technologies.

The Visible Ops Handbook: Implementing ITIL in 4 Practical and Auditable Steps ( http://www.itpi.org/the-visible-ops-handbook-review.html )

Is a methodology designed to jumpstart implementation of controls and process improvement.

The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win ( http://itrevolution.com/books/phoenix-project-devops-book/ )

How DevOps techniques can fix the problems that happen in IT organizations.

Explaindio Video Creator Platinum 3.028 Multilingual | 487.53 MB

The No 1 Animation, Doodle Sketch, and Motion Video Creation Software. How To Create Attention-Grabbing Professional-Looking Marketing, Explainer & Training Videos in Just Minutes! Make Amazing Professional Animated, Doodle Sketch, and Motion Videos in Minutes!

CAPTIVATE ENGAGE EXPLAIN
Make Amazing Professional 2D & 3D Animated, Doodle Sketch, and Motion Videos in Minutes.

Easy & Intuitive
Just pick the object & action from library, and combine them with your own videos, images, and audio.

Match Your Message to a Style
2D & 3DAnimated, Doodle Sketch, Whiteboard, and Full Motion Videos with custom eye-catching transitions & on screen effects.

Unlimited Commercial Use Available
Get commercial license option to create videos also for your clients.

Video is an online reflection of your business and visitors judge your business based on experience from watching your video
Video has become the most powerful way for businesses and marketers to sell & engage with their target audiences online. But this is not just about having a video. Video is an online reflection of your business and visitors judge your business based on experience from watching your video. All it takes for your visitor to turn away from your website forever is a bad first impression caused by a non-appealing video.

Instantly grabbing visitors attention, engaging them, and building trust are the key factors of a powerful video.
Marketers, advertisers, trainers, and communicators from businesses both small and large all want to use more motion, animation, or doodle videos to attract and keep viewers attention, as well as to pass on their message in a powerful and effective way.

The only video content creation software
This is the only video content creation software that does all this, and it can literally send your conversion soaring sky high. After supporting tens of thousand of video creators, we have discovered that videos which combine animation, whiteboard sketch and full motion video have the biggest impact on the viewers attention and engagement.

With Explaindio even complete beginners can create professional quality
You do not need any video production experience to use Explaindio. Creating high quality, stand out video has never been this simple. Explaindio allows anyone, no matter what their experience level to create professional appealing videos.

Tweak video ads for better conversions
Looking to quickly change your video ads to tweak them for better conversions? Explaindio gives you that flexibility and freedom to do changes very quickly as well. There is currently no more affordable video content creator with this much flexibility.

More Profits In A Shorter Amount Of Time
Explaindio makes it possible to create attention-grabbing, professional-looking, marketing, advertising, and explainer videos in just minutes. That translates to more profits, in a shorter amount of time, without production cost, which could run up to ten of thousands of dollars, and without waiting for weeks or months to have the video produced.

Explaindio Ends Outrageous Costs And Long Production Times
Historically those outrageous costs and long production times have lead entrepreneurs or even bigger businesses to either abandon or significantly limit their video promotions. It all changes with Explaindio. Now marketers, advertisers, trainers, and communicators from businesses both small and large can use more videos to pass on their message, without the previous restraints.

What’s New in Explaindio 3:
– Multiple animations at the same time
– Full multi-timeline editing
– Full 3D animations
– Import of external 3D elements
– 3D models and animation customization
– 180+ pre-made text animation
– 140+ pre-made video, image, doodle objects animations
– Easy video creation wizard
– Keyframe animation
– New built-in transitions
– Import videos in most popular format like AVI, WMV, FLV, MOV, and MP4
– Youzign And Pixabay Integration

Download :
Anh em cần download phần mềm comment để lại email gởi email link download cho né

In this post, we’ll see how to configure the PPTP server on Mikrotik.

Login to the Mikrotik RouterOS via Winbox and go to the IP —> Pool:

Click on the “Plus Symbol” to add new pool, Name it whatever you want (like I named it pptp-pool), add the address range (192.168.10.10-192.168.10.20, in my case), then hit OK:

Now go to the PPP Section. Click “PPTP Server” and check the Enabled:

While still on the PPP window, switch to the “Profiles” tab. Click on the plus sign to create new profile, Name it whatever you want (like I use pptp-profile), set the pool that we have created earlier for “Local Address” AND “Remote address“, then press OK.

Now switch to the “Secrets” tab of the PPP window. Click on the plus sign to create new user, add the name(which act as username), password and profile that we have created in the previous step:

Click on the IP —> Firewall:

From the “Filter Rules” tab, add the new rule. Set the chain to input, protocol to tcp and Dst. port to 1723:

Switch to the Action tab and set it to accept, then Click OK:

Add another new rule. Set the chain to input and protocol to gre:

Switch to the Action tab and set it to accept, then Click OK:

Note: Drag these two rules ABOVE THE DEFAULT “drop” RULE

PPTP Client Setup on Windows 7:

From “Control Panel“, select the “Network and Sharing Center” and then choose “Set up a new connection or network“:

On the next window, choose “Connect to a workplace“:

Choose “Use my Internet Connection (VPN)” from the next window:

Type the Public IP address or hostname of your Mikrotik, on the next window:

Next type your VPN username/password and after that click Connect:

Connection usually takes a minute to connect, upon successfully connected, will show you the message “You are connected“:

Verify the pptp logs on Mikrotik by hitting Log:

Ping any internal host:

NOTE: Sometimes, there is a problem to access other hosts on the LAN from the VPN. The solution to this problem is to set up the proxy-arp on the local interface that connect to your LAN:

Hope this will help you!

Scenario:

Mikrotik is working as dhcp, dns, and default gateway for local network.

Extra Requirement:

Mikrotik only assign ip address(es) to authorized client(s) in our local network!

Here are the steps to achieve this requirement:

Connect to the Mikrotik router using it’s ip address through web browser:

Click on the … button, select the ip address of Mikrotik, enter username and password, then click Connect:

IP —>DHCP Server

From the Leases tab, select the client(s), which are authorized to take ip address from Mikrotik router (in future) and then click Make Static:

Move to the DHCP tab and double-click on the DHCP Server and select the static-only from Address Pool drop down menu:

After that, only authorized client(s) will get ip address from Mikrotik. If you want that new client get an ip address from Mikrotik, then you can select the dhcp_pool1 from Address Pool drop down menu. After that Mikrotik will assign an ip address to new client, make this ip address to static (as described above) and select static-only again in order to disallow ip address assignment to unknown client(s).

Hope this will help you!

Scenario:

Mikrotik dhcp server with 3 network cards (LAN,Wireless,WAN), WAN is connected to Internet, Wireless is connected to the 10.10.10.0/24 subnet and serve as dhcp server while LAN interface serve for 172.16.10.0/24 subnet.

Connect to the Mikrotik RouterOS using winbox/ssh and check the ip address(es) of the interfaces:

ip address print

Issue this command, in order to configure the dhcp server for LAN subnet:

ip dhcp-server setup

Issue the same command for the dhcp server configuration on Wireless interface:

ip dhcp-server setup

Now, check the ip setting of the client(s) machines on both subnets:

ipconfig /all

Also confirm the dhcp leases on Mikrotik:

IP -> DHCP Server -> Leases

Hope this will help you!

In this tutorial, we will use the Site-to-Site VPN scenario with the modification and one of the customer site that is using Mikrotik router, which is also acting as gateway for LAN plus the vpn gateway while from the AWS side, we are using the exact same Ubuntu Linux router.

Please review the previous tutorial before starting this tutorial, as I’ll use the previous tutorial as the base for this one.

Note: Please don’t waste your time in hacking, all these public devices and IP(s) are Temporary, I have destroyed them after finished this tutorial.

VPN Configuration on Mikrotik Site:

Open the IP->IPsec window in WinBox:

Create a new Proposal(if you don’t want to use the default) as follows:

Now, create a new policy as follows:

From the General Tab

Src Address: Mikrotik LAN 192.168.10.0/24 Subnet Dst Address: AWS VPC 10.100.0.0/16 Private Subnet

From the Action Tab:

SA Src Address: MIkrotik WAN Address 102.162.166.90 SA Dst Address: AWS VPC Linux NAT Router WAN Address 54.219.146.242 Tick the Tunnel checkbox For Proposal: Use LAN2AWSProposal or whatever proposal you have created in the first step.

Next, Move to the Peers tab and create a new peer by using the public address of AWS NAT Instance as Address:

Next, create a NAT Bypass rule, to exclude the AWS VPC Private Subnet(s) to be natted:

Placed the above created rule at the top of all other NAT rule(s) and clear the connection table from existing connection or reboot the Mikrotik.

VPN Configuration on AWS VPC:

Also allow the ICMP packet on internal subnet security group from the remote LAN for testing purpose:

Edit the ipsec.conf file:

vi /etc/ipsec.conf

Here is the addition to the ipsec.conf file (please refer to the ipsec.conf file from previous tutorial):

conn AWS2MikrotikConnection left=10.100.10.10 leftsubnets=10.100.0.0/16 leftid=54.219.146.242 leftsourceip=10.100.10.10 right=102.162.166.90 rightsubnets=192.168.10.0/24 rightid=102.162.166.90 pfs=no forceencaps=yes authby=secret auto=start

Edit the shared secret file:

vi /etc/ipsec.secrets

Mine ipsec.secrets file as an example:

Restart the IPSec service:

service ipsec restart

Verify the status of IPSec service on Ubuntu at AWS VPC:

service ipsec status

Note: Please don’t panic, just restart the service one more time if it didn’t come up.

Verify that the Traffic is passing through the Tunnel:

Ping from the AWS vpn gateway to the Mikrotik LAN IP:

Ping from AWS VPC private Subnet to Mikrotik’s LAN for verification:

Ping from the Local machine to the machine on VPC Private subnet:

VERY Useful Tip:

If the Tunnel didn’t come up after the configuration, just restart the server and also start the ping from your LAN host to other side LAN host.

In Some Scenarios , We May need to Block Facebook Social Website or some others …In this tutorial, I will show you, how to achieve this goal using L7 (Layer7).

This tutorial consists of two parts:

1- Block facebook website for everyone on local network.

2- Allow facebook for specific host(s) and block for others on local network.

1- Block facebook website for everyone on local network.

First we check that Facebook is currently working on our local network or not?

Check the IP address of our client?

We need to create new Regexp rule at Layer7 Protocols, in order to block the facebook for our local network.

To achieve this goal, please follow these steps:

^.+(facebook.com).*$

Now, we need to create Filter Rule, using these steps:

Now test the rule, that we just created:

Try also on 2nd client (172.16.10.199/24):

Check that it only block facebook or other websites also?

Oh yes, our rule is working perfectly

2- Allow facebook for specific host(s) and block for others on local network.

Now, we want to allow facebook for 2nd client (172.16.10.199/24) but still want to block it for other host(s).

To accomplish this goal, we need to create a second Filter rule, to do this, please follow these steps:

Move this rule at the top:

Test this rule on 2nd client (172.16.10.199/24):

Verify the rule on Mikrotik:

Verify that, facebook is still blocked for other host(s) on the local network:

Verify the rule(s) on Mikrotik:

Drop packets rate are incremented!

We can do the same for youtube or any other website!

We want to authenticate and authorize the user on Cisco devices using the Microsoft Active Directory. In this tutorial, we want to create two Active Directory groups, Network Admin and Network Tech. Network Admin have full access to Cisco devices (privilege level 15) while Network Tech have custom access (like show commands including show running and interface configurations) only.

arbab belong to the Network Admin group

ali belong to the Network Tech group

Network Policy and Access Services:

Before starting this tutorial, I assume that you have installed the Active Directory and it is up and running. Let’s start the tutorial by adding the NPS Role:

Start -> Administrative Tools -> Server Manager

Click on Roles and then select the Add Roles:

Click Next:

Select Network Policy and Access Services and click Next:

Click Next:

Select Network Policy Server and click Next:

Click Install:

Click Close:

Configure Cisco Router as RADIUS Client in NPS:

Right Click on RADIUS Client and Select New:

Fill the information according to your environment:

Router/Switch Friendly Name, IP address and Shared secret:

Create an Active Directory User Group:

Next, we need to create an Active Directory User Group, which will use to access the routers/switches.

 Start -> Administrative Tools -> Active Directory Users and Computers

Right click on User, and select Group from the New menu:

Name your Group, select the Group scope and type:

Right Click on the newly created Group and select Properties:

Move to the Members tab and click on Add:

Select the desired member that you want to add to this group and click OK:

After adding all the desired members, Finally click OK:

Create a Network Policy:

Right Click on Network Policy and select New:

Give any meaningful name and select Next:

Click on Add:

Select Windows Groups and click Add:

A popup window will be opened, click Add Groups on it:

Select the Group that we have created above (may be you want to use your existing group) and click OK:

Again Click OK:

After Adding the Group, Click Next:

Select Access granted and click Next:

Uncheck the default options and check Unencrypted authentication (PAP, SPAP) and select Next:

Select No on the popup window:

Skip the Constraints values by clicking Next (else you can configure it according to your requirement):

Remove the default attributes by selecting each attribute and click remove:

After that click on Add:

Select Service-Type from Attributes and click on OK:

Select Login from Attribute Information:

It will return you to the previous page, from there select Close:

Select Vendor Specific under RADIUS Attributes and click Add:

From the Vendor, Select Cisco and Cisco-AV-Pair from Attributes and Click Add:

It will open popup window, select Add on this window:

Enter shell:priv-lvl=15 for the Privilege level 15 user’s group (aka Network Admin with full access on the routers/switches) and click OK:

After returning to the RADIUS Attributes window, click Next:

Click Finish:

Move this Network Policy on the top:

Cisco Router Configuration:

Check the ip address and ping the Domain Controller to verify the connectivity:

show ip interface brief

Configure AAA security services,radius group and private key:

conf t aaa new-model aaa group server radius ADAAA server-private 192.168.179.250 key password

Note: ADAAA is just my group name and 192.168.179.250 is the ip address of Domain controller, so change them according to your environment.

To enable the Authentication & Authorization, use the following commands:

aaa authentication login default group ADAAA aaa authorization exec default group ADAAA

Now, configure the authentication to the vty line(s):

line vty 0 4 transport input telnet ssh login authentication default

Telnet/SSH to the Router from Client Machine:

Now, try to login to the router from the client machine using the Active Directory username and password:

Enter the username and password:

Success

Create new Network Policy with Privilege level 3:

Member of this group can only check the configuration using show commands and can only configure the interface(s).

Create a new Active Directory Group (same as we created above), assign the users to this group and also create the new network policy using the exact same steps that we follow above with the changes at the following steps:

shell:priv-lvl=3

Move this Network Policy on the top:

Test from Client Machine:

User Ali belong to the privilege level 3 group that can only run show command and even cannot check the running config:

Allow user Ali, to view the running config, configure the router using level 15 account or using the console:

privilege exec all level 3 show running-config

Now, check that Ali can view the running config:

show running-config view full

User Ali, even cannot run the configure terminal command:

Allow user Ali, to run the configure terminal command as well as the interface related commands:

privilege exec level 3 configure terminal privilege configure all level 3 interface

Now, run configure terminal and interface commands again as user Ali:

Hope this will help you!

In the event port forwarding is needed, a NAT Rule will need to be created in the Mikrotik.

For this example, the NAT Rule is to allow access to a device on IP 192.168.88.100 using port 80 (extension 100).

For devices such as onsite PBX that have remote extensions and need a range of ports, use a hyphen (example: 10000-20000).

To create the NAT rule, please do the following:

1. Log into the Mikrotik using Winbox and go to IP
2. Go to Firewall

3. From the Firewall window, go to the NAT tab
4. Click on the Blue Plus Sign to add a new rule
5. From the New NAT Rule window, under the General tab, set the following settings:
   1. Chain: dstnat
   2. Protocol: tcp
   3. Dst. Port: 8080 (to use a port range use a hyphen, example: 10000-20000)
   4. In. Interface: ether1-gateway
6. Click on Action tab

7. For Action set to dst-nat
8. To Addresses: 192.168.88.100 (example)
9. To Ports: 80 (to use a port range use a hyphen, example: 10000-20000)
10. Click Apply
11. Click Comment

12. In the Comment for NAT Rule <8080> add a comment to help identify the rule (e.g.: Ext 100)
13. Click OK to close the comment window
14. Click OK to close the NAT Rule window

15. The rule will now appear in bold to show that the rule is active

To make the rule inactive or to disable select the rule (the rule will be highlighted in blue) and click on the red "X" or type "d" to disable the rule.

** WARNING **

It is important to only have the rule active when working on the device. Do not leave the rule active when not working on the device or the device will get compromised.

How to make a Mikrotik act as a wireless station

*Purpose: In this case, we have a customer that uses a wireless cell modem for Internet access which lacks ethernet ports, but we would also like to give access to some devices that are not wifi enabled. This is our workaround for that problem.

Equipment: Any available wifi network and in our case a Mikrotik 751-2HnD

Step 1: Create a Security Profile

-Click Wireless

-Click Security Profiles

-Click red plus to add new profile

-Type Security Profile Name

-Select Access Point/Router Authentication Type(Encryption method)

-Select Unicast and Group Ciphers

-Type the password of the access point or wireless network and enter it into either the WPA or WPA2 based on the encryption method on the access point. If this is open wifi then leave fields blank.

Step 2: Delete wlan1 Bridge

-Click Bridge

-Click Ports Tab

-Right Click on wlan1 and remove

Step 3: Turn DHCP Client to wlan1

-Click IP

-Click DHCP Client

-Double click wlan1

-Change drop down menu to wlan1

-Click OK

Step 4: Change NAT "Out" Interface

-Click IP

-Click Firewall

-Click NAT Tab

-Change “Out” Interface to wlan1

Step 5: Change MikroTik to Station Mode

-Click Wireless

-Double click on wlan1

-Click the Wireless tab

-Change drop down menu Mode to station

-Change Drop Down menu Security Profile to the Security Profile Entered in step 1

Step 6: Perform Scan

-Click Scan

-Double Click the SSID of the Network you are connecting to.

You should now have internet.

By default Mikrotik use the following LAN subnet, 192.168.88.0/24 with the gateway IP being 192.168.88.1.

For this example the Mikrotik’s subnet is being changed to 10.0.0.0/24 with a gateway IP of 10.0.0.1.

There are six items that will be updated.

• Add the local address to the Mikrotik’s Bridge-Local
• Add the new IP range to the Mikrotik’s Pool
• Add the new address to the DHCP Network
• Change the DHCP Server to use the new Address Pool
• Change the DNS Static address
• Add the new address to the IP Service List’s Available From address.

Add the Local Address

1. Open Winbox and go to IP
2. Go to Addresses
3. In the Address List window, click on the Blue Plus Sign to add
4. In the New Address window, select Address to type in the new address
5. Type in the new address: 10.0.0.1/24
6. Select the Bridge-Local Interface
7. Click OK
8. The new address now appears in the Address List

Add the New IP Range

9. Go to IP
10. Go to Pool
11. In the IP Pool window, click on the Blue Plus Sign to add
12. Enter in a Name (e.g.: cust-pool)
13. Enter in the address pool range (e.g.: 10.0.0.10-10.0.0.99)
14. Click OK
15. The new IP Pool now appears in the IP Pool list

Add the New Address to the DHCP Network

16. Go to IP
17. Go to DHCP Server
18. In the DHCP Server window, click on the Blue Plus Sign to add
19. Enter in the Address (e.g.: 10.0.0.0/24)
20. Enter in the Gateway (e.g.: 10.0.0.1)
21. Enter in the DNS Servers (e.g.: 10.0.0.1)
22. Click OK
23. The new netwoeks will now appear in the DHCP Server Networks list

Change the DHCP Pool

24. From the same DHCP Server, double click on "default" to change the Address Pool
25. Go to the Address Pool drop down menu and select the pool that was just created (e.g.: cust-pool)
26. Click OK
27. The new Address Pool is now selected for the DHCP server

Change tge DNS Static Address

28. Go to IP
29. Go to DNS
30. From the DNS Settings window, click on Static
31. Select the default DNS Static entry by double clicking the entry
32. Change the address to the new Gateway IP (e.g.: 10.0.0.1)
33. Click OK
34. The new Address will now appear in the DNS Static list

Add the New Available From Address

35. Go to IP
36. Go to Services
37. From the IP Service List window, select the "www" entry by double clicking the entry
38. Click on the arrows to add a new entry
39. Enter the new subnet (e.g.: 10.0.0.0/24)
40. Click OK
41. The new Available From subnet will now appear on the IP Service List for the www service.

To access the mikrotik, please change your network settings to be able to access 192.168.88.1 which is the default IP address of the Mikrotik.

Once the network settings have been updated, please connect to ETH2 or port 2 to access the LAN connection. Please note that ETH1 or port 1 is for the WAN connection.

With the network settings able to access 192.168.88.1 and your computer or laptop is connected to eth2, please open a web browser and enter in 192.168.88.1 in the address bar.

You will now have the web login screen available. Here will want to download Winbox, which is the tool we will be using to manage the Mikrotik.

1. From the Mikrotik login webpage, click on Winbox to download the Winbox application.
2. Once Winbox has been downloaded, open the application and it will find the Mikrotik, if not press Refresh to find the Mikrotik.
3. The Mikrotik will display the Mikrotik with the MAC Address, IP Address, Identity, Version, and the Board.
4. By default the Login is admin and there is no password. Press Connect to connect to the Mikrotik.

The following will be configured on the Mikrotik:

• Set a password for the admin account
• Set an Identity on the Mikrotik
• Set an SSID for the wireless connection
• Set a wireless key to connect to the wireless connection
• Add a static WAN IP
• Add a gateway for the WAN IP
• Add DNS servers
• Add a Firewall Rule to allow remote access via Winbox
• Add a Firewall Rule to prevent DNS DoS Attack
• Disable SIP ALG and h323
• Disable Services: ftp, ssh, telnet
• Add an Available from address to match the LAN subnet
• Set the Clock to desired timezone
• Set SNTP Client
• Enable the Cloud feature

Set Password for Admin Account

5. Go to System
6. Go to Users
7. From the User List, select the admin account by double clicking the account
8. From the User <admin> window, click on Password
9. From the Change Password window, enter in the desired password and re-enter to confirm
10. Click OK from the Change Password window
11. Click OK from the User <admin> window

Set Identity

12. Go to System
13. Go to Identity
14. In the Identity window, enter in an identity to identifty the given Mikrotik (i.e.: Cust Name)
15. Click OK

Set an SSID

16. Go to Wireless
17. From the Wireless Tables select wlan1 by double clicking the wlan1 interface
18. From the Interface <wlan1> window, find SSID and enter in the desired SSID
19. Click OK

Set a Wireless Key

20. From the Wireless Tables window, go to the Security Profiles tab
21. From the list, find the default Security Profile and double clicking the default profile
22. From the Security Profile <default> window, Change the Mode to "dynamic keys"
23. From the Authentication Type, select WPA2 PSK by checking the box
24. In the WPA2 Pre-Shared Key, enter in the desired wireless key (if the key is too short, the WPA2 Pre-Shared Key text will be red. If acceptable, it will be blue)
25. Click OK

Add a static WAN IP

26. Go to IP
27. Go to Addresses
28. From the Address List window, click the blue Plus Sign
29. From the New Address window, enter in the WAN IP that will be used which is provided by the ISP (i.e.: 12.34.56.78/29)
30. Select the Interface and set to ether1-gateway
31. Click Apply
32. The Network feild will auto populate with the network address
33. Click OK
34. Note that the WAN IP now appears on the Address List window

Add a Gateway for WAN IP

35. Go to IP
36. Go to Routes
37. From the Route List window click the blue Plus Sign
38. From the New Route window, enter in the Gateway that was provided by the ISP (i.e.: 12.34.56.73)
39. Click OK
40. Note that the Gateway is now appears on the Route List window (when ETH1 is connected to the WAN connection, the ISP’s bridged modem, the status should be reachable)

Add DNS servers

41. Go to IP
42. Go to DNS
43. From the DNS Settings window, enter in the DNS server addresses provided by the ISP; click on the arrows to add a second server address (i.e.: 4.2.2.1, 4.2.2.2)
44. Press OK

Add Firewall Rule for Winbox

45. Go to IP
46. Go to Firewall
47. From the Firewall window, click the blue Plus Sign
48. From the New Firewall Rule, do the following:
    1. Select Chain, and set to input
    2. Select Protocol, and set to tcp
    3. Enter in the Dst. Port to 8291
    4. Select the In. Interface, and set to ether1-gateway
49. Go to the Action tab
50. Select Action, and set to accept
51. Click OK
52. Select the new rule which is now seen on the Filter Rules list
53. Drap the new rule to the top

Add Firewall Rule for DNS DoS Attack

54. From the Firewall window, click the blue Plus Sign
55. From the New Firewall Rule, do the following:
    1. Select Chain, and set to input
    2. Select Protocol, and set to udp
    3. Enter in the Dst. Port to 53
    4. Select the In. Interface, and set to ether1-gateway
56. Go to the Action tab
57. Select Action, and set to drop
58. Click OK
59. The new rule is now seen on the Filters Rule list

Disable SIP (SIP ALG)

60. From the Firewall window, click on the Service Ports tab
61. Select h323 and SIP (use Ctrl to select both)
62. Hit "d" on your keyboard or click the red "x" to disable, both h323 and sip now appear grayed out which means they are now disabled

Disable IP Services

63. Go to IP
64. Go to Services
65. Select the following Services from the Service List; ftp, ssh, and telnet
66. Hit "d" on your keyboard or click the red "x" to disable the services selected, they will now appear grayed out which means they have been disabled

Add an Available from Address

67. From the IP Services List, double click on "www" to edit the service
68. In the IP Service <www> window, type in the local LAN subnet (i.e.: 192.168.88.0/24)
69. Click OK
70. The available from address now appears with the LAN subnet.

Set Clock

71. Go to System
72. Go to Clock
73. From the Time Zone Name drop down menu, select the Time Zone for the device
74. Click OK

Set SNTP

75. Go to System
76. Go to SNTP Client
77. Check the box for Enable
78. In the Primary NTP server field enter in: time.nist.gov In the Secondary NTP server field enter in: us.pool.ntp.org
79. Click Apply.
80. If DNS was set, the server addresses will be changed to IPs (NTP Server text will be blue, if DNS is not set NTP Server address will bered)
81. Click OK

Enable Cloud Feature

82. Go to IP
83. Go to Cloud
84. Check the box for Enable
85. If the Mikrotik is connected to the internet, the Public Address and DNS Name will become populated (use the DNS Name to log in remotely regardless of the IP address)
86. Click OK

To set QoS or Quality of Service successfully, we first want to know the rates that the internet circuit is stable at.

– What does that mean?

When running a capacity speed test (such as speedtest.net), the results will reflect the speed of the circuit, however the circuit will not be stable at those speeds.

To find the stable speeds of the circuit, we recommend using an application speed test.

For the purposes of this knowledge article, we will assume that the application speed test results for the circuit, which we will be using for service, has 1.5 MB down and 1.5 MB up (this is just an example, results will differ for each internet circuit).

For this example, we will be using five phones (we allocate 100 KB per phone) which means that we will want to reserve 500 KB for all phones.

The remaining bandwidth, which is 1000 KB or 1MB will be used for data.

Lastly, voice will be given the highest priority and data the lowest priority.

• 500k for voice with highest priority
• 1M for data with lowest priority

– But, how does the Mikrotik know the difference between voice and data traffic?

Managle rules will be added that will mark the connection to and from our RTP proxy (192.92.8.0/27) and then mark those packets.

Việt nam như các anh em cũng biết là đất nước con rồng cháu tiên với nghề làm web truyền thống chỉ xoay quanh web nhạc, web phim, web tin tức.

Tôi đoán là web kiểu này dễ hút khách. Admin có thể len lén cài quảng cáo kiếm tí tiền, có thể là ly cà phê sáng hoặc hoá đơn tiền net tháng đéo nào cũng thu, dù cáp tiên rồng vẫn cứ hay đứt đúng lúc.

Web tin tức không nói, nhưng nhạc và phim có 1 vấn đề lớn về storage và bandwidth, mà ở VN này ko thể thu phí dc, có chó nó mua.

Anh em nên nhớ chúng ta có thể đọc 1 năm 1/2 cuốn sách, chứ phim thì bú đều ngày-3-bữa, nhất là phim, hehe, thôi không nói ngại quá. Nếu anh em mở web phim, giới mộ điệu không thiếu, có chăng là thiếu chỗ chứa phim (storage) và thiếu băng thông truyền tải (bandwidth).

Biên nhanh về storage: 1 bộ phim HAY chất lượng 720p khoảng tầm 1GB. Suy ra để làm 1 web phim cỡ vừa với trên dưới 10000 bộ, anh em cần 10TB storage.

Amazon cloud storage S3 lừng danh ai cũng biết tôi ko nói nữa. Họ thu phí lưu trữ 0.025 đỗ nam trung cho 1GB dữ liệu nhạy cảm, mà nếu ko nhạy cảm giá vẫn thế !

10 TB ta cần trả cho amazon 250$ tháng.

Đó là chưa kể giá băng thông, cứ cho trung bình 1 tháng 1 phim trong web dc coi 10 lần, ta cần băng thông 100 TB. Amazon S3 lừng danh gọi đây là Data Transfer, giá rẻ như cắt cổ gà 0.07.

Khỏi phải nói ta cần móc túi chi thêm cho tụi con buôn này 7000$ mỗi tháng.

Và anh em hãy nhớ lấy, tuy phải trả 7250$ nhưng nếu anh em thu phí xem phim, web của anh em sẽ ko 1 bóng người.

Giải pháp storage gần miễn phí

Google Drive! ai ai cũng biết google có chính sách khuyến học, chỉ cần là sinh viên với 1 email đuôi edu minh chứng, anh em sẽ có ngay 1 acc google drive với dung lượng ko hạn chế.

Thị trường chợ đen đang bán acc dạng này với giá nửa triệu việt nam đồng, nhưng hãy lưu ý hạn dùng của acc. Theo kinh nguyệt của tôi, thường là từ lúc mua cho đến lúc người bán lấy lại. ( có thể 1 ngày hoặc vài tháng – hên xui)

Nói thì nói, chúng ta là lập trình viên thần thánh hiển nhiên đã thất-học từ lâu, ko cần dùng email edu, anh em hãy chịu chi 10$ 1 tháng cho gói gsuite bussiness, đãi ngộ tương tự nếu ko nói là ngon hơn.

Như thế từ 250 đô 1 tháng cho storage, chúng ta chỉ tốn 10$.

Đó là chưa kể khi anh em up phim lên. Drive sẽ tự động xem ké, và convert phim gốc sang các thể loại nhẹ hơn, tỉ dụ anh em up phim 1080p thì sau đó anh em sẽ có phim 1080p, 720p, 480p. Các link phim giảm chất lượng này quá phù hợp để chíu trên mobile. Điểm hay nhất là drive tự làm FREE.

Như vậy giải pháp storage đã có. Giờ là vấn đề băng thông.

Hô biến drive thành streaming server

Hiển nhiên Drive không phải host để có thể stream video, mục đích của drive là để lưu trữ những thước phim tình cảm gia đình. Như phim con bé anh chị gọi Ba lần đầu, tất nhiên thể loại tình cảm gia đình mà anh chị vừa nghĩ đến cũng có thể up lên, nhưng tôi khuyên anh chị rất không nên up. Hãy nhớ internet never forgets and never forgives.

Khi đoạn phim tình cảm trên được up lên, drive sẽ cho các anh chị chuột phải và chọn Preview để xem. Một popup overlay sẽ hiện lên và chiếu lại đoạn phim nhạy cảm.

Là lập trình viên thần thánh, chúng ta ngay lập tức mở chrome dev tools và bắt dc 1 request URL như hình.

Dễ dàng nhận thấy link get_video_info nhận vào 1 param là docId, đây chính là id của đoạn phim tình cảm. Hãy xem response của request trên trả về nội dung gì

 status=ok&hl=en&allow_embed=0&ps=docs&partnerid=30...... 

Trông có vẻ là 1 query string, bắt đầu bằng status=OK, chúng ta có thể đoán nếu status !== OK là có lỗi.

Thử parse đoạn response dùng chính lib querystring của nodejs thì chúng ta thấy dc 1 thứ rất thú vị, ngay key fmt_stream_map đó là các link stream videoplayback theo kiểu số|link, ngăn cách nhau bởi dấu phẩy

Thử copy 1 link ra chạy thì aha đây là link stream ta đang tìm. Nhúng link này vào thẻ video của html ta có ngay 1 trang phim không tốn bandwidth.

Vậy còn cái số trước link là gì, ta thấy có các số 18, 22, 35 vân vân và mây mây. Nếu các anh em thành tâm muốn biết wikipedia sẽ trả lời:

Xem thêm: https://en.wikipedia.org/wiki/YouTube

Như vậy theo hình chụp từ trang wikipedia số 18 ý nói phim tình cảm được lưu là video/mp4, dc mã hoá chuẩn H.264 và chất lượng 360p.

Đến đây, anh em có thể code 1 tool input vào docId và output ra các link phim với đủ chuẩn loại và chất lượng khác nhau.

Bài viết đến đây là hết.

Đó nếu là người khác sẽ nói vậy nhưng tôi biên tut ko bao giờ giấu nghề hehe, hãy đọc tiếp vì chúng ta còn nhiều thứ hay ho để nói.

Giờ trước khi qua vấn đề kế, hãy cùng ôn lại những ưu điểm của link stream này:

1. Gắn vào thẻ video của html ngon lành, hỗ trợ controls play pause seek, tốc độ cực nhanh vì có cdn toàn cầu, tài trợ bởi google lão anh hùng và các internet provider local. (yes băng thông phim lấy từ link stream kia dc tài trợ bởi viettel, vdc và fpt đó, google ko tốn xu nào đâu)
2. Link stream videoplayback ko có id của phim tình cảm, giúp người up an tâm ngủ ngon và sống đúng với tuổi của mình ko cần xin phép ai như bạn Vàng Anh.
3. Từ link stream videoplayback chúng ta ko có cách nào suy ngược ra dc link gốc, ngay cả google cũng thế. Cũng ko thể báo cáo vi phạm giúp drive trở thành vùng đất an lành cho thể loại phim khó nói nhưng dễ hình dung. ( gần đây google có 1 vài động thái để debug vụ link này, sẽ bàn ở bên dưới)
4. Quan trọng nhất, anh em không tốn 7000 đô la đỗ nam trung bandwidth, do hot link trực tiếp từ server của google. Các anh em làm phim ở VN ko tốn 1 xu tiền bandwidth nào cho tới bi giờ.
5. Up 1 phim thôi nhưng lấy về nhiều link với nhiều loại chất lượng khác nhau. Tối ưu cho mobile.
6. Bypass dc limit của google drive. Cái này ko nhiều anh em biết, nhưng nếu anh em share 1 file từ google drive cho nhiều đồng nghiệp xem vui chơi giải trí. File đó nếu HAY sẽ bị block ngay, do nhiều view quá. Link stream trên coi thoải mái ko lo bị block.

OK, giờ khi đường tăng xin link đã thỉnh dc link, chúng ta hãy gởi link cho bạn bè xem để chảnh chó. Kết quả ko ai coi dc. Đây là lúc anh em nhận ra một sự thật:

Link stream videoplayback bị giới hạn bởi IP Address

Nói thì nói, link này là do anh em tự preview phim của mình để coi lại, hiển nhiên ai preview thì người đó coi. Google lão anh hùng đã nhúng kèm vào đoạn querystring IP của người lấy link, và check từng bit của ip đối chứng khi chíu phim, kết quả là những bạn bè đồng nghiệp có IP khác sẽ bị 403.

Giờ đây chúng ta đối mặt với 1 vấn đề khó khăn nhất, làm thể nào qua mặt dc google khi link bị giới hạn bởi IP của người GET, làm thế nào để chúng ta có thể GET link từ server của mình, để rồi gởi cho hàng triệu con dân đang gào thét vì đói phim tình cảm hoặc phim nhạy cảm ?

Qua mặt hệ thống ip mapping của google

Hãy xem lại định dạng link stream videoplayback, bên dưới là các querystring quan trọng mà các anh em cần lưu ý

https://r15---sn-i3b7knez.c.drive.google.com/videoplayback?id=2c6374eb1653ae1c itag=18 source=webdrive mime=video/mp4 ip=171.226.93.247 ipbits=0 expire=1486127839 

itag như đã nói, là một param do youtube (dc google mua lại, hiện thuộc sở hữu của google) tự đặt ra, hoàn toàn ko có document chính thống nào mô tả, ngay trong youtube api cũng ko có. Tuy nhiên các anh hùng cào phím trên toàn thế giới qua một thời gian dài quan sát đã tổng hợp lại trên wikipedia ( link ref cuối bài )

source nguồn của file, nhìn vào đây ta có thể đoán dc nguồn của phim tình cảm đang xem là ở drive hay youtube hay picasa.

ip ip của người/server GET.

ipbits false (0) hoặc bằng một số khác 0, nếu bằng false (0) nghĩa là google sẽ ko quan tâm đến ip server GET, tức là link này ai cũng có thể xem dc. Nếu anh em thắc mắc làm sao GET dc link có ipbits bằng 0, hãy đọc tiếp.

Chiêu số 1: 2 tay 2 IP ( IPv4 vs IPv6 )

Các anh hùng cào phím đã nghĩ ra một cách hết sức đơn giản, nếu server GET link có IPv6 còn người xem có Ipv4 thì google sẽ check ip bit như thế nào khi 2 giao thức (protocol) là hoàn toàn khác nhau ?

Thực tế Google ko check bit dc, do đó link GET dùng ipv6 thì tất cả client IPv4 đều xem được và ngược lại.

Áp dụng trò này, các trang phim sẽ detect IP của người xem, và dựa vào đó để trả về link stream tương ứng.

Công thức rất đơn giản là:

người xem có ipv4 => link phim ipv6

người xem có ipv6 => link phim ipv4

Chiêu số 2: lấy tiền đè người (google gsuite for business)

Nếu anh em có acc gsuite for business của google, thì hãy dùng acc đó để request link get_video_info, kết quả trả về có ipbits = 0 do đó video link get ra ai cũng có thể xem dc bất kể ipv4 hay v6, không cần phải kiểm tra ip của client nữa.

Áp dụng 1 trong 2 cách trên, chúng ta có thể qua mặt google về vụ giới hạn ip, tuy nhiên còn 1 vấn đề rất quan trọng là link get ra thỉnh thoảng có 1 số người không coi dc, tỉ lệ này rất ngẫu nhiên, vì sao ?

Qua mặt hệ thống Google Global Cache (GGC)

Lượng dữ liệu truyền đi từ server của google là rất lớn, do đó nó cần 1 hệ thống phân phối dữ liệu phân tán đi khắp các nơi trên thế giới. Đảm bảo khi user xem phim, phim đó phải dc lấy về từ 1 con server có tốc độ nhanh nhất và gần nhất đối với user đó, từa tựa CDN chắc anh em cũng chả lạ gì.

Nói 1 cách đơn giản, hệ thống này của google hoạt động như 3 bước bên dưới:

1. Các internet provider như viettel, fpt, vdc đưa 1 số máy chủ cho google quản lý. Các máy chủ này sẽ dc hoà vào mạng lưới google edge network và trở thành 1 trung tâm lưu cache cho phim tình cảm (GGC location)
2. Khi user truy cập, google sẽ detect IP và provider của user, các bạn ko đọc nhầm, google biết bạn đang dùng internet của viettel hay fpt, thậm chí biết dc chính xác gói cước internet của bạn.
3. Nếu dãy ip và provider của user này nằm trong GGC, nó sẽ redirect user về một trong các server lưu cache đó. Giờ hãy nhìn lại URL link phim mà các bạn lấy dc: https://r15---sn-i3b7knez.c.drive.google.com thấy chứ r15---sn-i3b7knez con server r15 này nằm ở malaysia vì mình dùng viettel gói viettel-sgn20, mà malaysia khỏi phải nói nằm kế bên nhau hehe cách nhau mổi cây cầu dừa.

Lý do 1 số đồng nghiệp không xem dc video nếu bạn gởi link trên qua là vì dãy ip và internet provider của đồng nghiệp khác mạng lưới GGC. Do đó bị 403.

Để qua mặt dc GGC, hay nói các khác bắt GGC redirect user về đúng location cache, chúng ta cần quay ngược thời gian lại năm 2005. Vào năm đó, google như bao thanh niên trẩu tre khác quyết định làm web phim.

Google mua domain googlevideo.com và hào hứng bừng bừng triển ngay dự án, kết quả tất nhiên là dự án chạy dặt dẹo và nhanh chóng khuất-núi. Sau đó, như các bạn đã biết google mua lại youtube.

Tuy nhiên một tài nguyên vô giá là domain redirector.googlevideo.com vẫn tồn tại, nó tiếp tục dc sử dụng và các anh hùng cào phím nhanh chóng nhận ra đây mà 1 mắt xích rất quan trọng trong google global cache. Nếu các bạn đá user về redirector.googlevideo.com, nó sẽ tự detect user ip và ngay lập tức redirect user về location đúng.

Đó là lý do sau khi lấy dc link video, chúng ta cần thay thế domain này https://r15---sn-i3b7knez.c.drive.google.com thành https://redirector.googlevideo.com để tận dụng và bắt GGC làm việc cho chúng ta.

Giờ nếu các bạn chưa tin và nghi ngờ GGC, hãy vào https://redirector.googlevideo.com/report_mapping để biết mình đang dùng internet của hãng nào, và tên gói cước internet đang dùng.

OK bài tut đã quá dài tuy nhiên chúng ta vẫn còn vấn đề cuối cùng.

Link stream videoplayback bị giới hạn bởi COOKIE

Có tất cả 3 endpoint get_video_info cho google drive mà các anh hùng cào phím trên internet tìm được

https://mail.google.com/e/get_video_info
https://drive.google.com/e/get_video_info
https://docs.google.com/e/get_video_info

trong đó ngoại trừ link đầu tiên là mail vẫn còn dùng dc, 2 cái phía sau đều bị dính COOKIE. Vậy dính cookie là sao ?

Khi chúng ta gọi https://drive.google.com/e/get_video_info, trong response header trả về có header set-cookie, có 1 cookie name là DRIVE_STREAM chứa value dùng để playback video, ai gọi thì lụm dc cookie này.

Nếu chúng ta gởi link video lấy dc cho user khác, hiển nhiên máy user đó ko có cookie DRIVE_STREAM dẫn đến videoplayback sẽ bị lỗi 403.

Trước đây cả 3 link đều chạy tốt, giờ còn 1, đây là động thái của google nhằm chặn đường các web phim ?
Cũng cần phải nói link mail cũng chạy rất chập chờn, 10 lần gọi dc vài lần thôi.

Ngoài ra, cũng mới update gần đây link videplayback dùng get_video_info gọi ra có kèm 1 param mới là driveid chứa id thật của video phim tình cảm. Chỉ cần biết id này các hãng có thể report abuse với google để tận diệt các phim vi phạm bản quyền, đều mà trước đây họ ko làm dc vì link redirector ko có id, và lại còn bị expired sau một thời gian nên họ có report thì google cũng ko giải quyết.

Rất nhiều web phim ở vn đang bị lộ driveid do update mới này.
Lại một lần nữa chúng ta thấy google đang thể hiện quyết tâm tiêu diệt các web phim.

Kết bài

Bài đã quá dài, nên tác giả ngưng viết đi nhậu. Nếu đề tài này được anh em ủng hộ và quan tâm tác giả sẽ viết phần 2 với nhiều cảnh hot hơn.

Phần 2: Giải pháp get link google drive không bao giờ die

Đặc biệt không sợ google fix lỗi, không sống bằng lòng tốt của google, link get ra xem được trên mọi thiết bị và còn chống chôm link

Cùng sự ra mắt của 1 dịch vụ mới: Đường Tăng Xin Link.

Chào thân ái và quyết thắng !